-- Logs begin at Wed 2022-07-20 21:44:52 UTC, end at Wed 2022-07-20 23:19:47 UTC. -- Jul 20 22:24:48.500265 dv-cinder-799871-38-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 20 22:24:48.507648 dv-cinder-799871-38-iscsi devstack@keystone.service[80434]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 20 22:24:48.509654 dv-cinder-799871-38-iscsi devstack@keystone.service[80434]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Jul 20 22:24:48 2022] *** Jul 20 22:24:48.509654 dv-cinder-799871-38-iscsi devstack@keystone.service[80434]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 20 22:24:48.509654 dv-cinder-799871-38-iscsi devstack@keystone.service[80434]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 20 22:24:48.509654 dv-cinder-799871-38-iscsi devstack@keystone.service[80434]: nodename: dv-cinder-799871-38-iscsi Jul 20 22:24:48.509654 dv-cinder-799871-38-iscsi devstack@keystone.service[80434]: machine: x86_64 Jul 20 22:24:48.509654 dv-cinder-799871-38-iscsi devstack@keystone.service[80434]: clock source: unix Jul 20 22:24:48.509654 dv-cinder-799871-38-iscsi devstack@keystone.service[80434]: pcre jit disabled Jul 20 22:24:48.510110 dv-cinder-799871-38-iscsi devstack@keystone.service[80434]: detected number of CPU cores: 4 Jul 20 22:24:48.510110 dv-cinder-799871-38-iscsi devstack@keystone.service[80434]: current working directory: / Jul 20 22:24:48.510110 dv-cinder-799871-38-iscsi devstack@keystone.service[80434]: detected binary path: /usr/bin/uwsgi-core Jul 20 22:24:48.510110 dv-cinder-799871-38-iscsi devstack@keystone.service[80434]: your processes number limit is 47680 Jul 20 22:24:48.510110 dv-cinder-799871-38-iscsi devstack@keystone.service[80434]: your memory page size is 4096 bytes Jul 20 22:24:48.510110 dv-cinder-799871-38-iscsi devstack@keystone.service[80434]: detected max file descriptor number: 2048 Jul 20 22:24:48.510110 dv-cinder-799871-38-iscsi devstack@keystone.service[80434]: lock engine: pthread robust mutexes Jul 20 22:24:48.510110 dv-cinder-799871-38-iscsi devstack@keystone.service[80434]: thunder lock: enabled Jul 20 22:24:48.510110 dv-cinder-799871-38-iscsi devstack@keystone.service[80434]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 20 22:24:48.510110 dv-cinder-799871-38-iscsi devstack@keystone.service[80434]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Jul 20 22:24:48.555243 dv-cinder-799871-38-iscsi devstack@keystone.service[80434]: Python main interpreter initialized at 0x562c532f79c0 Jul 20 22:24:48.555243 dv-cinder-799871-38-iscsi devstack@keystone.service[80434]: python threads support enabled Jul 20 22:24:48.555243 dv-cinder-799871-38-iscsi devstack@keystone.service[80434]: your server socket listen backlog is limited to 100 connections Jul 20 22:24:48.555243 dv-cinder-799871-38-iscsi devstack@keystone.service[80434]: your mercy for graceful operations on workers is 90 seconds Jul 20 22:24:48.555621 dv-cinder-799871-38-iscsi devstack@keystone.service[80434]: mapped 403077 bytes (393 KB) for 2 cores Jul 20 22:24:48.555697 dv-cinder-799871-38-iscsi devstack@keystone.service[80434]: *** Operational MODE: preforking *** Jul 20 22:24:48.555976 dv-cinder-799871-38-iscsi devstack@keystone.service[80434]: *** uWSGI is running in multiple interpreter mode *** Jul 20 22:24:48.555976 dv-cinder-799871-38-iscsi devstack@keystone.service[80434]: spawned uWSGI master process (pid: 80434) Jul 20 22:24:48.556188 dv-cinder-799871-38-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 20 22:24:48.561284 dv-cinder-799871-38-iscsi devstack@keystone.service[80434]: spawned uWSGI worker 1 (pid: 80440, cores: 1) Jul 20 22:24:48.561284 dv-cinder-799871-38-iscsi devstack@keystone.service[80434]: spawned uWSGI worker 2 (pid: 80441, cores: 1) Jul 20 22:24:48.561284 dv-cinder-799871-38-iscsi devstack@keystone.service[80434]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 20 22:24:54.065571 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 20 22:24:54.065571 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: from cryptography.utils import int_from_bytes Jul 20 22:24:54.066409 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 20 22:24:54.066409 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: from cryptography.utils import int_from_bytes Jul 20 22:24:54.066529 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 20 22:24:54.066529 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: from cryptography.utils import int_from_bytes Jul 20 22:24:54.066529 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 20 22:24:54.066529 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: from cryptography.utils import int_from_bytes Jul 20 22:24:54.115123 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 20 22:24:54.115339 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 20 22:24:54.115434 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 20 22:24:54.115531 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] command line args: [] {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 20 22:24:54.115611 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 20 22:24:54.115732 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] command line args: [] {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 20 22:24:54.115816 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 20 22:24:54.115964 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ================================================================================ {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 20 22:24:54.116056 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 20 22:24:54.116195 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] ================================================================================ {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 20 22:24:54.116281 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] admin_token = **** {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 22:24:54.116453 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] admin_token = **** {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 22:24:54.116533 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] config_dir = [] {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 22:24:54.116648 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 22:24:54.116735 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] config_dir = [] {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 22:24:54.116834 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] config_source = [] {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 22:24:54.116913 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 22:24:54.117017 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] debug = True {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 22:24:54.117103 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] config_source = [] {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 22:24:54.117219 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] debug = True {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 22:24:54.117304 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 22:24:54.117542 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 22:24:54.117675 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 22:24:54.117780 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 22:24:54.117887 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 22:24:54.117985 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 22:24:54.118086 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 22:24:54.118187 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 22:24:54.118297 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 22:24:54.118405 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 22:24:54.118508 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] list_limit = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 22:24:54.118609 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] list_limit = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 22:24:54.118705 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] log_config_append = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 22:24:54.118811 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] log_config_append = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 22:24:54.118913 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 22:24:54.119053 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 22:24:54.119166 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] log_dir = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 22:24:54.119275 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] log_dir = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 22:24:54.119379 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] log_file = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 22:24:54.119486 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] log_file = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 22:24:54.119599 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 22:24:54.119683 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 22:24:54.119806 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 22:24:54.119883 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 22:24:54.120004 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 22:24:54.120087 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 22:24:54.120203 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 22:24:54.120293 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 22:24:54.120373 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 22:24:54.120508 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 22:24:54.120625 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 22:24:54.120709 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 22:24:54.120794 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 22:24:54.120872 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 22:24:54.120997 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 22:24:54.121075 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 22:24:54.121198 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 22:24:54.121274 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 22:24:54.121400 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 22:24:54.121483 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 22:24:54.121560 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 22:24:54.121682 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 22:24:54.121758 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 22:24:54.121879 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 22:24:54.121964 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 22:24:54.122107 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 22:24:54.122215 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 22:24:54.122358 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 22:24:54.122491 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 22:24:54.122636 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 22:24:54.123542 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.81/identity {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 22:24:54.123714 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.81/identity {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 22:24:54.123793 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] publish_errors = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 22:24:54.123929 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] publish_errors = False {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 22:24:54.124013 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 22:24:54.124150 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 22:24:54.124235 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 22:24:54.124330 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 22:24:54.124414 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 22:24:54.124577 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 22:24:54.124661 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 22:24:54.124813 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 22:24:54.124904 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 22:24:54.125048 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] standard_threads = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 22:24:54.125132 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 22:24:54.125281 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 22:24:54.125367 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] standard_threads = False {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 22:24:54.125524 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 22:24:54.125638 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 22:24:54.125754 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 22:24:54.125835 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 22:24:54.125905 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] use_journal = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 22:24:54.125999 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 22:24:54.126077 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] use_json = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 22:24:54.126221 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] use_stderr = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 22:24:54.126295 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] use_journal = False {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 22:24:54.126366 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] use_syslog = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 22:24:54.126441 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] use_json = False {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 22:24:54.126570 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] use_stderr = False {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 22:24:54.126657 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 22:24:54.126763 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] use_syslog = False {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 22:24:54.126853 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.127033 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 22:24:54.127163 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.127317 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.127397 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.127554 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.127635 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.127806 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.127884 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.128043 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.128128 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.128289 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.128368 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.128525 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.128602 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.128770 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.128863 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.129015 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.129101 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.129263 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.129348 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.129497 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.129582 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.129742 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.129828 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.130102 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.130188 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.130267 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.130344 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.130428 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.130577 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.130665 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.130840 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.130989 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.131132 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.131226 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.131402 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.131488 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.131635 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.131712 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.131890 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.132154 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.132241 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.132343 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.132423 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.132500 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.132622 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.132710 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.132853 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.132988 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.133073 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.133266 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.133357 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.133519 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.133598 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.133753 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.133838 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.133997 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.134086 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.134228 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.134307 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.134457 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.134536 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.134701 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.134816 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.134946 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.135102 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.135205 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.135309 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.135443 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.135557 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.135656 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.135897 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.135984 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.136129 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.136221 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.136358 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.136438 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.136586 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.136741 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.136831 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.136980 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.137068 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.137221 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.137306 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.137461 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.137540 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.137695 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.137781 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.137929 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.138017 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.138156 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.138284 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.138401 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.139091 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.139221 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.139304 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.139481 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.139560 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.139713 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.139796 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.139952 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.140033 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.140185 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.140303 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.140419 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.140498 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.140654 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.140737 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.140889 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.140966 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.141127 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.141206 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.141366 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.141447 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.141598 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.141675 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.141832 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.141917 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.142062 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.142143 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.142298 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.142384 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.142530 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.142609 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.142759 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.142844 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.142993 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.143135 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.143247 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.143327 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.143486 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.143587 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.143743 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.143824 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.143993 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.144113 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.144268 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.144344 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.144456 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.144541 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.144690 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.144769 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.144925 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.145006 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.145185 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.145263 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.145426 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.145504 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.145665 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.145753 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.145907 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.145984 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.146145 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.146222 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.146380 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.146459 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.146613 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.146699 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.146852 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.146932 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.147112 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.147205 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.147357 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.147437 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.147597 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.147675 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.147836 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.147923 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.148078 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.148179 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.148308 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.148393 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.148542 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.148629 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.148778 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.148857 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.149014 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.149099 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.149243 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.149333 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.149476 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.149562 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.149700 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.149788 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.149911 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.150005 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.150140 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.150222 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.150380 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.150461 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.150609 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.150696 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.150842 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.150926 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.151100 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.151182 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.151340 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.151431 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.151572 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.151652 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.151816 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.151895 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.152052 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] auth.external = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.152139 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.152283 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.152362 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.152526 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.152605 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.152788 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.152875 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.153016 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] auth.password = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.153095 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.153245 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] auth.token = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.153350 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.153484 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.153565 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] auth.external = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.153718 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.153809 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.153962 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.154040 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.154217 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.154296 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.154464 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.154552 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] auth.password = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.154697 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] auth.token = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.154776 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.154929 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.155058 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.155187 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] credential.caching = True {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.155284 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.155433 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.155519 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.155666 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.155747 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.155911 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.155999 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.156147 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.156227 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.156363 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.156446 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.156599 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] credential.caching = True {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.156690 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.156848 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.156929 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.157097 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.157176 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.157331 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.157419 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.157571 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.157651 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.157818 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.157900 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.158053 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.158208 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.158317 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.158399 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.158572 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.158652 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.158811 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.158901 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.159067 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] federation.caching = True {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.159151 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.159308 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.159974 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.160138 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.160227 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.160309 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.160382 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.160454 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.160525 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.160603 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.160684 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.160760 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.160832 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] federation.caching = True {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.160910 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.160983 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.161063 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.161144 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.161316 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.161412 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.161487 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.161566 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.161648 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.161722 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.161839 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] identity.caching = True {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.161924 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.162085 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.162165 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.162332 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.162414 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.162567 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.162646 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.162801 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.162888 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.163132 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.163217 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.163372 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] identity.caching = True {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.163453 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.163621 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.163707 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.163871 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.163952 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.164108 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.164194 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.164341 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.164420 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.164575 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.164704 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.164818 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.164942 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.165065 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.165183 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.165312 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.165436 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.165560 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.165691 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.165816 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.165934 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.166063 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.166169 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.166318 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.166409 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.166560 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.166640 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.166800 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.166881 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.167047 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.167133 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.167292 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.167372 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.167525 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.167606 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.167761 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.167875 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.168004 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.168116 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.168245 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.168356 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.168493 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.168573 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.168724 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.168804 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.168970 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.169051 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.169218 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.169300 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.169454 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.169533 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.170234 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.170336 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.170483 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.170567 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.170712 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.170791 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.170953 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.171070 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.171207 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.171294 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.171448 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.171566 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.171683 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.171764 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.171924 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.172003 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.172154 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.172240 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.172389 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.172469 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.172864 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.172949 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.173113 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.173205 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.173345 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.173424 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.173575 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.173654 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.173812 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.173905 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.174051 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.174132 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.174286 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.174364 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.174520 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.174609 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.174755 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.174840 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.174985 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.175133 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.user = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.175245 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.175338 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.175497 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.175577 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.175729 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.175807 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.175968 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.176059 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.176205 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.176285 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.176438 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] ldap.user = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.176518 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.176678 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.176767 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.176906 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.176993 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.177141 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.177221 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.177373 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.177460 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.177608 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.177703 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.177860 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.177959 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.178079 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.178194 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.178294 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.178390 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.178534 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.178641 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.178769 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.178891 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.178995 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.179159 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.179256 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.179352 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.179488 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.179607 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.179729 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.179835 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.179969 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.180074 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.180207 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.180321 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.180445 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.180550 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.180681 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.180784 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.180928 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.181041 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.181164 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.181275 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.181403 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.181509 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.181637 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.181817 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.181927 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.182023 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.182119 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.182213 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.182315 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.182420 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.182553 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.182658 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.182800 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.182915 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.183113 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.183210 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.183356 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.183436 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.183594 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.183675 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.183828 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.183917 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] resource.caching = True {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.184069 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.184149 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.184314 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.184397 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.184551 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.184640 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.184781 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.184899 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.185022 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.185104 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.185275 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.185392 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] resource.caching = True {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.186089 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.186173 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.186343 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.186423 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.186581 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.186671 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.186822 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] role.caching = True {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.186901 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.187098 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] role.driver = sql {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.187183 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.187332 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.188669 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.188669 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.188949 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.189026 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.189098 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.189180 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.189255 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.189328 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] role.caching = True {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.189446 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.189545 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] role.driver = sql {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.189641 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.189747 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.189852 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.189951 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.190049 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.190145 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.190242 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.190348 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.190447 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.190544 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.190642 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.190738 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.190835 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.190939 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.191058 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.191161 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.191259 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.191368 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.191464 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.191574 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.191674 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.191771 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.191867 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.191964 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.192060 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.192165 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.192264 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.192361 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.192460 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.192557 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.192654 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.192758 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.192857 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.192960 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.193059 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.193156 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.193261 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.193360 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.193457 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.193561 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.194214 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.194302 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.194378 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.194450 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.194529 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.194602 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.194673 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.194753 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.194828 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.194900 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.194971 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.195083 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.195160 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.195248 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.195323 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.195395 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.195466 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] token.caching = True {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.195537 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.195608 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.195687 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.195761 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.195887 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.195970 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.196140 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.196220 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.196377 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.196459 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.196613 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.196693 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] token.caching = True {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.196852 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.196931 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.197093 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.197176 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.197331 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.197411 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.197567 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.197648 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.197807 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.197889 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.198045 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.198125 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.198288 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.198367 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.198533 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.198615 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.198763 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.198842 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.199031 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.199115 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.199269 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.199358 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.199506 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.199587 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] database.connection = **** {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.199754 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.199845 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.200004 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.200092 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.200236 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.200317 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.200481 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.200561 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.200714 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.200825 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.201476 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.201546 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] database.connection = **** {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.201720 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.201880 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.201958 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.202030 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.202190 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.202280 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.202428 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.202508 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.202680 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.202760 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.202916 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.203027 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.203188 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.203268 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.203426 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.203505 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.203670 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.203763 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.203903 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.203983 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.204146 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.204226 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.204379 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.204467 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.204542 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80440) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 20 22:24:54.204614 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.204854 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.205101 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.205333 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.205576 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.205818 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 22:24:54.206044 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80441) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 20 22:24:54.235896 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 22:24:54.236126 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 22:24:54.236355 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 22:24:54.237313 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 22:24:54.237558 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 22:24:54.237790 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 22:24:54.244698 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 22:24:54.248228 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 22:24:54.272939 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 22:24:54.273173 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 22:24:54.274496 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 22:24:54.274735 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 22:24:54.423208 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 22:24:54.423759 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 22:24:54.472922 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 22:24:54.473158 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 22:24:54.473619 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 22:24:54.473862 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 22:24:54.548607 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 22:24:54.548852 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 22:24:54.549059 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 22:24:54.549204 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 22:24:54.549356 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 22:24:54.549518 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 22:24:54.549649 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 22:24:54.549777 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 22:24:54.549914 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 22:24:54.550037 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 22:24:54.550184 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 22:24:54.550184 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 22:24:54.579959 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 22:24:54.580192 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 22:24:54.580328 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 22:24:54.580460 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 22:24:54.580613 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 22:24:54.580743 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 22:24:54.580868 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 22:24:54.580993 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 22:24:54.581117 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 22:24:54.581250 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 22:24:54.581386 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 22:24:54.581512 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 22:24:54.581635 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 22:24:54.581635 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 22:24:54.593156 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 22:24:54.593739 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 22:24:54.606271 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 22:24:54.606482 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 22:24:54.606610 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 22:24:54.606735 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 22:24:54.606873 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 22:24:54.607040 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 22:24:54.607196 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 22:24:54.607321 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 22:24:54.607446 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 22:24:54.607577 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 22:24:54.607702 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 22:24:54.607825 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 22:24:54.607948 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80441) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 22:24:54.608136 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80440) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 22:24:54.625230 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.625424 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 20 22:24:54.625634 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.625775 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.625908 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 20 22:24:54.626038 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 20 22:24:54.626180 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.626310 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.626437 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 20 22:24:54.626566 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.626693 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.626828 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.626956 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.627121 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.627252 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.627382 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.638093 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.647617 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.647705 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.647776 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.647851 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.647924 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.648011 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.648011 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.648143 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80441) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 22:24:54.648211 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80440) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 22:24:54.651033 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80441) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 22:24:54.651303 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80440) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 22:24:54.651390 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.651583 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.651667 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.651865 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.651978 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.652264 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.652427 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.652511 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.652584 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.652780 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.652872 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.653043 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.653124 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.653310 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.653391 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.653583 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.653666 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.653853 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.653934 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.654143 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.681052 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.681167 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.682628 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80441) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 22:24:54.682728 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80440) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 22:24:54.682907 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.682988 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.690981 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80441) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 22:24:54.691091 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80440) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 22:24:54.691167 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.691241 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.691312 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.691396 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.696674 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80441) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 22:24:54.696766 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80440) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 22:24:54.696957 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.697039 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.702190 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80440) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 22:24:54.702376 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80441) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 22:24:54.702548 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.702843 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.703137 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.703511 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.703714 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.703846 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.703922 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.703994 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.703994 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.704205 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.704560 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.704836 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.705113 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.705382 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.723056 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.723336 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.724176 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.724452 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.727974 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80441) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 22:24:54.729076 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80440) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 22:24:54.731596 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.731889 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.732800 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.733095 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.736972 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80441) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 22:24:54.738133 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80440) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 22:24:54.740663 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80441) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 22:24:54.741801 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80440) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 22:24:54.744522 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.744824 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.745118 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.745405 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.745595 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.745680 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.745891 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.745977 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.746175 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.746466 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.746751 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.747046 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.766327 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80441) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 22:24:54.766708 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.766983 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.767234 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80440) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 22:24:54.774626 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.774759 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.774759 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.774759 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.775267 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.776084 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.776983 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.777263 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.777793 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.778069 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.780406 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80441) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 22:24:54.780692 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.780980 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.781202 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80440) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 22:24:54.781286 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.781484 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.781567 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.781766 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.782045 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.782328 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.795045 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80441) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 22:24:54.795384 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.795616 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.795813 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80440) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 22:24:54.795897 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.796093 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.796177 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.796374 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.796468 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.796653 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.796736 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.796930 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.797205 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.797478 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.816973 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80441) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 22:24:54.817602 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80440) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 22:24:54.820312 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80441) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 22:24:54.820933 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80440) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 22:24:54.823676 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.824285 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.825297 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.825982 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.827536 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80441) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 22:24:54.827822 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.828114 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.828206 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80440) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 22:24:54.828431 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.828720 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.835670 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.836200 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.837437 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80441) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 22:24:54.837822 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80440) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 22:24:54.840771 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.841063 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.841148 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.841352 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.841446 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.841662 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.841753 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.841938 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.851124 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80441) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 22:24:54.851281 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80440) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 22:24:54.851415 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.851565 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.851699 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.851842 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.859715 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80441) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 22:24:54.859813 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80440) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 22:24:54.862795 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.862877 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.862950 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.863053 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.863130 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.863202 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.863282 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.863358 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.863430 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.863503 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.863575 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.863655 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.863729 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.863802 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.863873 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.863945 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.864016 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.864092 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.864167 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.864239 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.864310 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.864382 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.864527 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.864603 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.864675 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80441) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.864747 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80440) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 22:24:54.902318 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x562c532f79c0 pid: 80441 (default app) Jul 20 22:24:54.902443 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x562c532f79c0 pid: 80440 (default app) Jul 20 22:24:54.905755 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42540197-0626-43f1-b8ca-1f4c6c1106c9 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:24:54.906054 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42540197-0626-43f1-b8ca-1f4c6c1106c9 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:24:54.906338 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42540197-0626-43f1-b8ca-1f4c6c1106c9 None None] PATH_INFO: `/v3` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:24:54.910652 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1/1] 10.222.0.81 () {50 vars in 791 bytes} [Wed Jul 20 22:24:54 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:25:03.437499 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a6be13-5947-4ecd-87f4-b7727e775fea None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:03.437997 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a6be13-5947-4ecd-87f4-b7727e775fea None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:03.438080 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a6be13-5947-4ecd-87f4-b7727e775fea None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:03.442421 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1/2] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:25:03 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:25:03.447747 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5731e3-fc4a-4a41-9d20-a3d1f9c5bd08 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:03.448169 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5731e3-fc4a-4a41-9d20-a3d1f9c5bd08 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:03.448253 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5731e3-fc4a-4a41-9d20-a3d1f9c5bd08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:03.515896 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: INFO dbcounter [None req-8c5731e3-fc4a-4a41-9d20-a3d1f9c5bd08 None None] Registered counter for database keystone Jul 20 22:25:03.537213 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG oslo_db.sqlalchemy.engines [None req-8c5731e3-fc4a-4a41-9d20-a3d1f9c5bd08 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80441) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 20 22:25:03.543337 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writer thread running {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 20 22:25:03.720488 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG passlib.handlers.bcrypt [None req-8c5731e3-fc4a-4a41-9d20-a3d1f9c5bd08 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80441) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 20 22:25:03.720831 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG passlib.handlers.bcrypt [None req-8c5731e3-fc4a-4a41-9d20-a3d1f9c5bd08 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80441) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 20 22:25:03.776618 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-8c5731e3-fc4a-4a41-9d20-a3d1f9c5bd08 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:25:03.781182 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-8c5731e3-fc4a-4a41-9d20-a3d1f9c5bd08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:25:03.900969 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 2/3] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:25:03 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 454 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 20 22:25:04.058812 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a6be13-5947-4ecd-87f4-b7727e775fea None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:04.127291 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: INFO dbcounter [None req-f4a6be13-5947-4ecd-87f4-b7727e775fea None None] Registered counter for database keystone Jul 20 22:25:04.147086 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG oslo_db.sqlalchemy.engines [None req-f4a6be13-5947-4ecd-87f4-b7727e775fea None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80440) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 20 22:25:04.153247 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writer thread running {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 20 22:25:04.270960 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a6be13-5947-4ecd-87f4-b7727e775fea None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:04.275281 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bace52-4726-4078-9f35-e06efaab2e77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:04.275944 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bace52-4726-4078-9f35-e06efaab2e77 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:04.276234 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bace52-4726-4078-9f35-e06efaab2e77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:04.276515 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bace52-4726-4078-9f35-e06efaab2e77 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:04.285161 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bace52-4726-4078-9f35-e06efaab2e77 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:04.337120 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: WARNING keystone.common.rbac_enforcer.enforcer [None req-27bace52-4726-4078-9f35-e06efaab2e77 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 20 22:25:04.337409 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bace52-4726-4078-9f35-e06efaab2e77 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:04.347481 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 2/4] 10.222.0.81 () {58 vars in 1324 bytes} [Wed Jul 20 22:25:04 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 20 22:25:04.353500 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5731e3-fc4a-4a41-9d20-a3d1f9c5bd08 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:04.368356 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5731e3-fc4a-4a41-9d20-a3d1f9c5bd08 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:04.373034 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d545240-01eb-45bf-994a-2a597c3f5487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:04.374265 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d545240-01eb-45bf-994a-2a597c3f5487 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:04.374556 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d545240-01eb-45bf-994a-2a597c3f5487 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:04.374838 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d545240-01eb-45bf-994a-2a597c3f5487 admin admin] PATH_INFO: `/v3/projects/015b89c7582349b2a89f8bbaea747017` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:04.376366 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d545240-01eb-45bf-994a-2a597c3f5487 admin admin] RBAC: Authorizing `identity:get_project(project_id=015b89c7582349b2a89f8bbaea747017)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:04.430338 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: WARNING keystone.common.rbac_enforcer.enforcer [None req-4d545240-01eb-45bf-994a-2a597c3f5487 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 20 22:25:04.430683 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d545240-01eb-45bf-994a-2a597c3f5487 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:04.435398 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 3/5] 10.222.0.81 () {56 vars in 1207 bytes} [Wed Jul 20 22:25:04 2022] GET /identity/v3/projects/015b89c7582349b2a89f8bbaea747017 => generated 344 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:25:06.461852 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a75864-5f75-4406-a9a6-89d3e01bb434 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:06.462338 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a75864-5f75-4406-a9a6-89d3e01bb434 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:06.462338 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a75864-5f75-4406-a9a6-89d3e01bb434 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:06.466490 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 3/6] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:25:06 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:25:06.471809 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aca575-b036-4d95-9dd1-1a55c6dd0efe None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:06.472249 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aca575-b036-4d95-9dd1-1a55c6dd0efe None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:06.472371 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aca575-b036-4d95-9dd1-1a55c6dd0efe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:06.491248 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-67aca575-b036-4d95-9dd1-1a55c6dd0efe None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:25:06.492754 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-67aca575-b036-4d95-9dd1-1a55c6dd0efe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:25:06.501202 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 4/7] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:25:06 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 20 22:25:06.523131 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a75864-5f75-4406-a9a6-89d3e01bb434 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:06.535658 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a75864-5f75-4406-a9a6-89d3e01bb434 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:06.540010 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cd4398-7e13-47f9-bc1b-00fa1abf334a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:06.540647 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cd4398-7e13-47f9-bc1b-00fa1abf334a admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:06.540939 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cd4398-7e13-47f9-bc1b-00fa1abf334a admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:06.541222 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cd4398-7e13-47f9-bc1b-00fa1abf334a admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:06.549516 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cd4398-7e13-47f9-bc1b-00fa1abf334a admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:06.551332 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cd4398-7e13-47f9-bc1b-00fa1abf334a admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:06.561005 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 4/8] 10.222.0.81 () {58 vars in 1324 bytes} [Wed Jul 20 22:25:06 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 22:25:06.566294 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67aca575-b036-4d95-9dd1-1a55c6dd0efe None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:06.575949 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67aca575-b036-4d95-9dd1-1a55c6dd0efe None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:06.580639 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05885c03-6250-45b7-babe-213e6efbc070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:06.581546 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05885c03-6250-45b7-babe-213e6efbc070 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:06.581834 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05885c03-6250-45b7-babe-213e6efbc070 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:06.582114 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05885c03-6250-45b7-babe-213e6efbc070 admin admin] PATH_INFO: `/v3/users/98f5d4c4be4845478aa0d31a1403d775` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:06.583615 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05885c03-6250-45b7-babe-213e6efbc070 admin admin] RBAC: Authorizing `identity:get_user(user_id=98f5d4c4be4845478aa0d31a1403d775)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:06.585485 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05885c03-6250-45b7-babe-213e6efbc070 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:06.587159 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 5/9] 10.222.0.81 () {56 vars in 1198 bytes} [Wed Jul 20 22:25:06 2022] GET /identity/v3/users/98f5d4c4be4845478aa0d31a1403d775 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:25:08.685918 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5254fa-2e39-4c3c-9aed-2f0f337a2691 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:08.686434 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5254fa-2e39-4c3c-9aed-2f0f337a2691 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:08.686516 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5254fa-2e39-4c3c-9aed-2f0f337a2691 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:08.687878 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 5/10] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:25:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:25:08.693059 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372b1642-16c0-4d2e-bb3f-ba27e028b8a5 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:08.693486 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372b1642-16c0-4d2e-bb3f-ba27e028b8a5 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:08.693636 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372b1642-16c0-4d2e-bb3f-ba27e028b8a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:08.710071 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-372b1642-16c0-4d2e-bb3f-ba27e028b8a5 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:25:08.711284 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-372b1642-16c0-4d2e-bb3f-ba27e028b8a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:25:08.723190 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e098ad-c5fd-4fb7-ad86-84efbfceacc0 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:08.723480 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e098ad-c5fd-4fb7-ad86-84efbfceacc0 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:08.723759 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e098ad-c5fd-4fb7-ad86-84efbfceacc0 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:08.725088 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 6/11] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:25:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 22:25:08.730059 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a3325c-042d-4fc2-b533-fbd071d4a7d6 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:08.730350 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a3325c-042d-4fc2-b533-fbd071d4a7d6 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:08.730625 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a3325c-042d-4fc2-b533-fbd071d4a7d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:08.770693 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG passlib.handlers.bcrypt [None req-58a3325c-042d-4fc2-b533-fbd071d4a7d6 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80440) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 20 22:25:08.771143 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG passlib.handlers.bcrypt [None req-58a3325c-042d-4fc2-b533-fbd071d4a7d6 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80440) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 20 22:25:08.795325 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 6/12] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:25:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 22:25:08.818122 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-58a3325c-042d-4fc2-b533-fbd071d4a7d6 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:25:08.818971 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372b1642-16c0-4d2e-bb3f-ba27e028b8a5 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:08.819547 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-58a3325c-042d-4fc2-b533-fbd071d4a7d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:25:08.842680 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:08.917467 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 7/13] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:25:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 22:25:08.922728 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372b1642-16c0-4d2e-bb3f-ba27e028b8a5 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:08.940332 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a3325c-042d-4fc2-b533-fbd071d4a7d6 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:08.956830 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf2cb88-cc13-47ce-9641-9e35e4f372cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:08.957643 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf2cb88-cc13-47ce-9641-9e35e4f372cf None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:08.957935 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf2cb88-cc13-47ce-9641-9e35e4f372cf None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:08.958218 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf2cb88-cc13-47ce-9641-9e35e4f372cf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:08.969268 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf2cb88-cc13-47ce-9641-9e35e4f372cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 22:25:08.980966 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-fbf2cb88-cc13-47ce-9641-9e35e4f372cf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 22:25:08.980966 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:25:08.980966 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:25:08.980966 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:25:08.980966 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:25:08.980966 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:25:08.980966 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:25:08.980966 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:25:08.980966 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:25:08.980966 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:25:08.980966 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:25:08.980966 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:25:08.980966 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 22:25:08.980966 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 22:25:08.980966 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 22:25:08.980966 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 22:25:08.981818 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 22:25:08.981818 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 22:25:08.981818 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 22:25:08.981818 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 22:25:08.981818 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:25:08.981818 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:25:08.981818 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:25:08.981818 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:25:08.981818 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:25:08.981818 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:25:08.981818 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:25:08.981818 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:25:08.981818 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:25:08.981818 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:25:08.981818 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:25:08.981818 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:25:08.987934 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:25:08.987934 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:25:08.987934 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:25:08.987934 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:25:08.987934 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 22:25:08.987934 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 22:25:08.987934 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 22:25:08.987934 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 22:25:08.987934 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 22:25:08.987934 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 22:25:08.987934 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 22:25:08.987934 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:25:08.987934 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 7/14] 10.222.0.81 () {56 vars in 1096 bytes} [Wed Jul 20 22:25:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 22:25:08.988577 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf2cb88-cc13-47ce-9641-9e35e4f372cf None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:09.039385 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a3325c-042d-4fc2-b533-fbd071d4a7d6 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:09.042085 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:09.070707 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:09.076538 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500ad227-eb37-498a-b75c-68916dfdad8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:09.077441 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500ad227-eb37-498a-b75c-68916dfdad8f None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:09.077731 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500ad227-eb37-498a-b75c-68916dfdad8f None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:09.078086 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500ad227-eb37-498a-b75c-68916dfdad8f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:09.088695 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf2cb88-cc13-47ce-9641-9e35e4f372cf None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:09.121364 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97166599-38f7-4278-b632-990ab77a6639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:09.122537 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97166599-38f7-4278-b632-990ab77a6639 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:09.122828 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97166599-38f7-4278-b632-990ab77a6639 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:09.123140 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97166599-38f7-4278-b632-990ab77a6639 None admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:09.123891 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97166599-38f7-4278-b632-990ab77a6639 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:25:09.124203 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97166599-38f7-4278-b632-990ab77a6639 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:09.125653 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97166599-38f7-4278-b632-990ab77a6639 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:09.138102 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 8/15] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Jul 20 22:25:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 22:25:09.143781 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97166599-38f7-4278-b632-990ab77a6639 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:09.146512 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500ad227-eb37-498a-b75c-68916dfdad8f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:09.148090 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500ad227-eb37-498a-b75c-68916dfdad8f None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:09.247261 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97166599-38f7-4278-b632-990ab77a6639 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:09.252909 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 8/16] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Jul 20 22:25:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 22:25:09.259308 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500ad227-eb37-498a-b75c-68916dfdad8f None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:09.262956 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:09.282989 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5eb5839-d1ea-4c63-b6b3-0f3fc817abc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:09.283921 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eb5839-d1ea-4c63-b6b3-0f3fc817abc3 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:09.284206 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eb5839-d1ea-4c63-b6b3-0f3fc817abc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:09.284486 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eb5839-d1ea-4c63-b6b3-0f3fc817abc3 None admin] PATH_INFO: `/v3/users/98f5d4c4be4845478aa0d31a1403d775` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:09.285695 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5eb5839-d1ea-4c63-b6b3-0f3fc817abc3 None admin] RBAC: Authorizing `identity:get_user(user_id=98f5d4c4be4845478aa0d31a1403d775)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:09.287190 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5eb5839-d1ea-4c63-b6b3-0f3fc817abc3 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:09.288813 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 9/17] 10.222.0.81 () {56 vars in 1177 bytes} [Wed Jul 20 22:25:09 2022] GET /identity/v3/users/98f5d4c4be4845478aa0d31a1403d775 => generated 247 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 22:25:09.294084 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5eb5839-d1ea-4c63-b6b3-0f3fc817abc3 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:09.308580 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:09.358122 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500ad227-eb37-498a-b75c-68916dfdad8f None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:09.391758 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5505c536-f861-463f-893d-0c41a1306ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:09.392577 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5505c536-f861-463f-893d-0c41a1306ae6 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:09.392856 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5505c536-f861-463f-893d-0c41a1306ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:09.393154 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5505c536-f861-463f-893d-0c41a1306ae6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:09.394522 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5eb5839-d1ea-4c63-b6b3-0f3fc817abc3 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:09.413035 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5505c536-f861-463f-893d-0c41a1306ae6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:09.414531 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5505c536-f861-463f-893d-0c41a1306ae6 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:09.429192 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e691e5-3060-4b9c-a68d-71aa0129fa86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:09.429997 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e691e5-3060-4b9c-a68d-71aa0129fa86 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:09.430284 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e691e5-3060-4b9c-a68d-71aa0129fa86 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:09.430562 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e691e5-3060-4b9c-a68d-71aa0129fa86 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:09.431776 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e691e5-3060-4b9c-a68d-71aa0129fa86 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:09.433315 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e691e5-3060-4b9c-a68d-71aa0129fa86 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:09.435524 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 10/18] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Jul 20 22:25:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 22:25:09.436749 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-5505c536-f861-463f-893d-0c41a1306ae6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:25:09.436749 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:25:09.436749 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:25:09.436749 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:25:09.436749 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:25:09.436749 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:25:09.436749 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:25:09.436749 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:25:09.436749 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:25:09.436749 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:25:09.436749 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:25:09.436749 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:25:09.436749 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 22:25:09.436749 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 22:25:09.436749 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 22:25:09.437697 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 22:25:09.437697 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:25:09.437697 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:25:09.437697 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:25:09.437697 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:25:09.437697 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:25:09.437697 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:25:09.437697 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:25:09.437697 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:25:09.437697 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:25:09.437697 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:25:09.437697 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:25:09.437697 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:25:09.437697 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:25:09.437697 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:25:09.437697 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:25:09.437697 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:25:09.438690 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 22:25:09.438690 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 22:25:09.438690 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:25:09.438690 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:25:09.438690 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 9/19] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Jul 20 22:25:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 22:25:09.441153 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e691e5-3060-4b9c-a68d-71aa0129fa86 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:09.444809 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5505c536-f861-463f-893d-0c41a1306ae6 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:09.500109 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:09.521780 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:09.540705 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e691e5-3060-4b9c-a68d-71aa0129fa86 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:09.544220 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5505c536-f861-463f-893d-0c41a1306ae6 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:09.579185 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e96a32-fc3c-41df-814d-a8a71a15d13b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:09.579827 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e96a32-fc3c-41df-814d-a8a71a15d13b None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:09.580115 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e96a32-fc3c-41df-814d-a8a71a15d13b None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:09.580398 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e96a32-fc3c-41df-814d-a8a71a15d13b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:09.581234 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e96a32-fc3c-41df-814d-a8a71a15d13b None admin] RBAC: Adding query filter params (role.id=d89b6c92d35a401a8517f1d47f6be843, scope.domain.id=default, user.id=98f5d4c4be4845478aa0d31a1403d775) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:25:09.581552 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e96a32-fc3c-41df-814d-a8a71a15d13b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:09.583028 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e96a32-fc3c-41df-814d-a8a71a15d13b None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:09.586207 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7ce642-7ec7-4cb0-904d-1c6ac8a3b0b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:09.586932 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7ce642-7ec7-4cb0-904d-1c6ac8a3b0b0 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:09.587278 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7ce642-7ec7-4cb0-904d-1c6ac8a3b0b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:09.587561 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7ce642-7ec7-4cb0-904d-1c6ac8a3b0b0 None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:09.588292 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7ce642-7ec7-4cb0-904d-1c6ac8a3b0b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:25:09.588608 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7ce642-7ec7-4cb0-904d-1c6ac8a3b0b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:09.589828 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 11/20] 10.222.0.81 () {56 vars in 1322 bytes} [Wed Jul 20 22:25:09 2022] GET /identity/v3/role_assignments?user.id=98f5d4c4be4845478aa0d31a1403d775&scope.domain.id=default&role.id=d89b6c92d35a401a8517f1d47f6be843 => generated 233 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:25:09.589984 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7ce642-7ec7-4cb0-904d-1c6ac8a3b0b0 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:09.610627 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 10/21] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Jul 20 22:25:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 22:25:11.843895 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b246329-5dc0-4dd3-a362-1a1e248c2661 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:11.844408 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b246329-5dc0-4dd3-a362-1a1e248c2661 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:11.844481 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b246329-5dc0-4dd3-a362-1a1e248c2661 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:11.846101 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 12/22] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:25:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:25:11.852850 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98013348-bd1f-4a8a-b20d-4cbb80f361d3 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:11.853252 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98013348-bd1f-4a8a-b20d-4cbb80f361d3 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:11.853434 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98013348-bd1f-4a8a-b20d-4cbb80f361d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:11.880044 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-98013348-bd1f-4a8a-b20d-4cbb80f361d3 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:25:11.881271 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-98013348-bd1f-4a8a-b20d-4cbb80f361d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:25:11.964343 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 11/23] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:25:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 22:25:12.008486 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b246329-5dc0-4dd3-a362-1a1e248c2661 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:12.143368 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b246329-5dc0-4dd3-a362-1a1e248c2661 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:12.183366 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f76ede-2f2d-4024-9125-301d8c678216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:12.184752 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f76ede-2f2d-4024-9125-301d8c678216 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:12.185031 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f76ede-2f2d-4024-9125-301d8c678216 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:12.187530 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f76ede-2f2d-4024-9125-301d8c678216 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:12.196186 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f76ede-2f2d-4024-9125-301d8c678216 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 22:25:12.204795 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-78f76ede-2f2d-4024-9125-301d8c678216 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 22:25:12.204795 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:25:12.204795 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:25:12.204795 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:25:12.204795 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:25:12.204795 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:25:12.204795 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:25:12.204795 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:25:12.204795 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:25:12.204795 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:25:12.204795 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:25:12.204795 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:25:12.204795 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 22:25:12.204795 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 22:25:12.204795 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 22:25:12.204795 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 22:25:12.205637 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 22:25:12.205637 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 22:25:12.205637 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 22:25:12.205637 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 22:25:12.205637 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:25:12.205637 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:25:12.205637 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:25:12.205637 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:25:12.205637 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:25:12.205637 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:25:12.205637 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:25:12.205637 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:25:12.205637 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:25:12.205637 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:25:12.205637 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:25:12.205637 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:25:12.206811 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:25:12.206811 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:25:12.206811 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:25:12.206811 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:25:12.206811 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 22:25:12.206811 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 22:25:12.206811 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 22:25:12.206811 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 22:25:12.206811 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 22:25:12.206811 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 22:25:12.206811 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 22:25:12.206811 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:25:12.206811 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 13/24] 10.222.0.81 () {56 vars in 1096 bytes} [Wed Jul 20 22:25:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:25:12.216670 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98013348-bd1f-4a8a-b20d-4cbb80f361d3 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:12.265330 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a987d8c1-d319-4be7-94de-521a1cd4b43b None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:12.265662 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a987d8c1-d319-4be7-94de-521a1cd4b43b None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:12.265954 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a987d8c1-d319-4be7-94de-521a1cd4b43b None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:12.267525 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 14/25] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:25:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:25:12.272893 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592818a8-7c15-435d-9554-9104d54d6a43 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:12.273436 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592818a8-7c15-435d-9554-9104d54d6a43 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:12.273726 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592818a8-7c15-435d-9554-9104d54d6a43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:12.281457 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:12.304887 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-592818a8-7c15-435d-9554-9104d54d6a43 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:25:12.306175 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-592818a8-7c15-435d-9554-9104d54d6a43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:25:12.310966 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:12.403788 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 15/26] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:25:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 22:25:12.404852 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1676f4-0c33-4c9c-b18a-4de76c4a94c9 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:12.405147 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1676f4-0c33-4c9c-b18a-4de76c4a94c9 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:12.405436 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1676f4-0c33-4c9c-b18a-4de76c4a94c9 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:12.406946 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 16/27] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:25:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:25:12.419250 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666e1888-2d5b-4e5a-9028-3a996b42813e None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:12.419537 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666e1888-2d5b-4e5a-9028-3a996b42813e None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:12.419821 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666e1888-2d5b-4e5a-9028-3a996b42813e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:12.447833 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98013348-bd1f-4a8a-b20d-4cbb80f361d3 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:12.459552 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-666e1888-2d5b-4e5a-9028-3a996b42813e None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:25:12.460764 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-666e1888-2d5b-4e5a-9028-3a996b42813e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:25:12.517827 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2544ad6a-d196-47b5-b736-e1d663c50a57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:12.518523 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2544ad6a-d196-47b5-b736-e1d663c50a57 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:12.518806 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2544ad6a-d196-47b5-b736-e1d663c50a57 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:12.519109 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2544ad6a-d196-47b5-b736-e1d663c50a57 None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:12.519866 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2544ad6a-d196-47b5-b736-e1d663c50a57 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:25:12.520176 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2544ad6a-d196-47b5-b736-e1d663c50a57 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:12.521575 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2544ad6a-d196-47b5-b736-e1d663c50a57 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:12.553278 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 12/28] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Jul 20 22:25:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:25:12.553897 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2544ad6a-d196-47b5-b736-e1d663c50a57 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:12.580375 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 17/29] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:25:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 22:25:12.581220 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666e1888-2d5b-4e5a-9028-3a996b42813e None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:12.644912 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:12.683919 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666e1888-2d5b-4e5a-9028-3a996b42813e None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:12.701085 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2544ad6a-d196-47b5-b736-e1d663c50a57 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:12.713844 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:12.723960 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15e652d-c5ac-4ee8-ac2d-deef58799bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:12.725556 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15e652d-c5ac-4ee8-ac2d-deef58799bbe None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:12.725556 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15e652d-c5ac-4ee8-ac2d-deef58799bbe None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:12.725556 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15e652d-c5ac-4ee8-ac2d-deef58799bbe None admin] PATH_INFO: `/v3/users/98f5d4c4be4845478aa0d31a1403d775` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:12.726805 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15e652d-c5ac-4ee8-ac2d-deef58799bbe None admin] RBAC: Authorizing `identity:get_user(user_id=98f5d4c4be4845478aa0d31a1403d775)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:12.728409 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15e652d-c5ac-4ee8-ac2d-deef58799bbe None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:12.730144 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 18/30] 10.222.0.81 () {56 vars in 1177 bytes} [Wed Jul 20 22:25:12 2022] GET /identity/v3/users/98f5d4c4be4845478aa0d31a1403d775 => generated 247 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:25:12.732024 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29d5c80-f4fa-4db5-a19a-4601fa57d9b3 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:12.732427 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29d5c80-f4fa-4db5-a19a-4601fa57d9b3 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:12.732780 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29d5c80-f4fa-4db5-a19a-4601fa57d9b3 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:12.734082 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 19/31] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:25:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:25:12.734863 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29d5c80-f4fa-4db5-a19a-4601fa57d9b3 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:12.750478 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d320ab0-4377-498e-bd24-bef400574a35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:12.751498 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d320ab0-4377-498e-bd24-bef400574a35 None admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:12.751793 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d320ab0-4377-498e-bd24-bef400574a35 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:12.752076 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d320ab0-4377-498e-bd24-bef400574a35 None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:12.753098 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d320ab0-4377-498e-bd24-bef400574a35 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:12.754660 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d320ab0-4377-498e-bd24-bef400574a35 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:12.846284 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29d5c80-f4fa-4db5-a19a-4601fa57d9b3 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:12.881036 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:12.889637 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b156fc-8d71-4e9d-bcb6-1d49d725ef5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:12.891107 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b156fc-8d71-4e9d-bcb6-1d49d725ef5a None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:12.891614 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b156fc-8d71-4e9d-bcb6-1d49d725ef5a None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:12.892175 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b156fc-8d71-4e9d-bcb6-1d49d725ef5a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:12.910599 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b156fc-8d71-4e9d-bcb6-1d49d725ef5a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:12.913349 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b156fc-8d71-4e9d-bcb6-1d49d725ef5a None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:12.950668 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-b2b156fc-8d71-4e9d-bcb6-1d49d725ef5a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:25:12.950668 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:25:12.950668 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:25:12.950668 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:25:12.950668 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:25:12.950668 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:25:12.950668 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:25:12.950668 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:25:12.950668 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:25:12.950668 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:25:12.950668 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:25:12.950668 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:25:12.950668 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 22:25:12.950668 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 22:25:12.950668 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 22:25:12.952437 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 22:25:12.952437 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:25:12.952437 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:25:12.952437 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:25:12.952437 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:25:12.952437 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:25:12.952437 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:25:12.952437 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:25:12.952437 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:25:12.952437 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:25:12.952437 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:25:12.952437 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:25:12.952437 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:25:12.952437 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:25:12.952437 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:25:12.952437 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:25:12.952437 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:25:12.954526 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 22:25:12.954526 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 22:25:12.954526 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:25:12.954526 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:25:12.954526 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 20/32] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Jul 20 22:25:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:25:12.954526 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b156fc-8d71-4e9d-bcb6-1d49d725ef5a None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:12.976144 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 13/33] 10.222.0.81 () {60 vars in 1131 bytes} [Wed Jul 20 22:25:12 2022] POST /identity/v3/roles => generated 219 bytes in 423 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:25:12.978392 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e870d17-10fe-48fc-913c-caa61d7405ed None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:12.978698 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e870d17-10fe-48fc-913c-caa61d7405ed None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:12.978982 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e870d17-10fe-48fc-913c-caa61d7405ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:13.003642 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-1e870d17-10fe-48fc-913c-caa61d7405ed None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:25:13.005406 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-1e870d17-10fe-48fc-913c-caa61d7405ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:25:13.081188 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 14/34] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:25:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 22:25:13.082152 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e870d17-10fe-48fc-913c-caa61d7405ed None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:13.125866 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b156fc-8d71-4e9d-bcb6-1d49d725ef5a None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:13.204709 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22012d8d-d8cd-43f7-9bac-38e00d2fc0bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:13.205449 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22012d8d-d8cd-43f7-9bac-38e00d2fc0bd None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:13.206190 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22012d8d-d8cd-43f7-9bac-38e00d2fc0bd None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:13.206626 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22012d8d-d8cd-43f7-9bac-38e00d2fc0bd None admin] PATH_INFO: `/v3/domains/default` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:13.209704 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22012d8d-d8cd-43f7-9bac-38e00d2fc0bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:13.211576 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22012d8d-d8cd-43f7-9bac-38e00d2fc0bd None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:13.213450 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 21/35] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Jul 20 22:25:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:25:13.214396 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22012d8d-d8cd-43f7-9bac-38e00d2fc0bd None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:13.227507 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:13.236262 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e870d17-10fe-48fc-913c-caa61d7405ed None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:13.279604 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:13.284963 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4996ac-4ce8-46ca-9a1a-92dfeb2782e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:13.286005 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4996ac-4ce8-46ca-9a1a-92dfeb2782e9 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:13.286439 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4996ac-4ce8-46ca-9a1a-92dfeb2782e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:13.287197 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4996ac-4ce8-46ca-9a1a-92dfeb2782e9 None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:13.288348 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4996ac-4ce8-46ca-9a1a-92dfeb2782e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:25:13.288822 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4996ac-4ce8-46ca-9a1a-92dfeb2782e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:13.291271 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4996ac-4ce8-46ca-9a1a-92dfeb2782e9 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:13.307719 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 15/36] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Jul 20 22:25:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:25:13.309567 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4996ac-4ce8-46ca-9a1a-92dfeb2782e9 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:13.348568 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22012d8d-d8cd-43f7-9bac-38e00d2fc0bd None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:13.388342 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaa6ab1-7900-4144-819b-0e269be95264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:13.389337 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaa6ab1-7900-4144-819b-0e269be95264 None admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:13.389809 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaa6ab1-7900-4144-819b-0e269be95264 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:13.390097 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaa6ab1-7900-4144-819b-0e269be95264 None admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:13.390849 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaaa6ab1-7900-4144-819b-0e269be95264 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:13.392296 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaaa6ab1-7900-4144-819b-0e269be95264 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:13.433819 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4996ac-4ce8-46ca-9a1a-92dfeb2782e9 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:13.468861 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342978dd-8bd6-45bd-b6ea-fa4601349b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:13.469680 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342978dd-8bd6-45bd-b6ea-fa4601349b8a None admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:13.469969 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342978dd-8bd6-45bd-b6ea-fa4601349b8a None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:13.470250 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342978dd-8bd6-45bd-b6ea-fa4601349b8a None admin] PATH_INFO: `/v3/domains/default/users/98f5d4c4be4845478aa0d31a1403d775/roles/d89b6c92d35a401a8517f1d47f6be843` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:13.472714 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342978dd-8bd6-45bd-b6ea-fa4601349b8a None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=98f5d4c4be4845478aa0d31a1403d775, role_id=d89b6c92d35a401a8517f1d47f6be843)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:13.474294 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342978dd-8bd6-45bd-b6ea-fa4601349b8a None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:13.544525 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 16/37] 10.222.0.81 () {58 vars in 1361 bytes} [Wed Jul 20 22:25:13 2022] PUT /identity/v3/domains/default/users/98f5d4c4be4845478aa0d31a1403d775/roles/d89b6c92d35a401a8517f1d47f6be843 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:25:13.546139 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8037de-2b7b-4ecd-8128-59ece9bb86c2 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:13.546781 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8037de-2b7b-4ecd-8128-59ece9bb86c2 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:13.547232 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8037de-2b7b-4ecd-8128-59ece9bb86c2 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:13.551061 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 17/38] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:25:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:25:13.551061 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8037de-2b7b-4ecd-8128-59ece9bb86c2 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:13.588039 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:13.685082 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 22/39] 10.222.0.81 () {60 vars in 1131 bytes} [Wed Jul 20 22:25:13 2022] POST /identity/v3/roles => generated 225 bytes in 471 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:25:13.686820 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bdf720-1add-4f93-b2f4-5fa226045f23 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:13.687150 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bdf720-1add-4f93-b2f4-5fa226045f23 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:13.687445 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bdf720-1add-4f93-b2f4-5fa226045f23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:13.709632 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8037de-2b7b-4ecd-8128-59ece9bb86c2 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:13.726195 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-71bdf720-1add-4f93-b2f4-5fa226045f23 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:25:13.727871 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-71bdf720-1add-4f93-b2f4-5fa226045f23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:25:13.785009 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0076cc62-3bfd-45f5-b179-6e42202bbc13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:13.785716 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0076cc62-3bfd-45f5-b179-6e42202bbc13 None admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:13.786033 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0076cc62-3bfd-45f5-b179-6e42202bbc13 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:13.786291 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0076cc62-3bfd-45f5-b179-6e42202bbc13 None admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:13.787120 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0076cc62-3bfd-45f5-b179-6e42202bbc13 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:13.788831 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0076cc62-3bfd-45f5-b179-6e42202bbc13 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:13.820928 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 23/40] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:25:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 22:25:13.822313 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e10c044-d316-4a7a-b53f-4160051b2f4b None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:13.822611 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e10c044-d316-4a7a-b53f-4160051b2f4b None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:13.822898 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e10c044-d316-4a7a-b53f-4160051b2f4b None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:13.824833 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 24/41] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:25:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:25:13.832708 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97fff98-cdb2-48d5-924d-d3f035eb74e7 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:13.833007 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97fff98-cdb2-48d5-924d-d3f035eb74e7 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:13.833364 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97fff98-cdb2-48d5-924d-d3f035eb74e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:13.854916 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-c97fff98-cdb2-48d5-924d-d3f035eb74e7 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:25:13.856245 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-c97fff98-cdb2-48d5-924d-d3f035eb74e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:25:13.903564 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:13.907593 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 18/42] 10.222.0.81 () {60 vars in 1141 bytes} [Wed Jul 20 22:25:13 2022] POST /identity/v3/projects => generated 301 bytes in 358 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:25:13.908310 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0076cc62-3bfd-45f5-b179-6e42202bbc13 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:13.939164 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 25/43] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:25:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 22:25:13.964879 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97fff98-cdb2-48d5-924d-d3f035eb74e7 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:14.010612 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:14.052630 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0076cc62-3bfd-45f5-b179-6e42202bbc13 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:14.090659 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97fff98-cdb2-48d5-924d-d3f035eb74e7 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:14.100286 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc57edeb-d54d-4091-8686-0d9a9c0282e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:14.101199 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc57edeb-d54d-4091-8686-0d9a9c0282e6 None admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:14.101302 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc57edeb-d54d-4091-8686-0d9a9c0282e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:14.102192 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc57edeb-d54d-4091-8686-0d9a9c0282e6 None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:14.102962 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc57edeb-d54d-4091-8686-0d9a9c0282e6 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:14.104682 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc57edeb-d54d-4091-8686-0d9a9c0282e6 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:14.134112 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1617a87-8969-4711-ae1e-6f6430114c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:14.134939 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1617a87-8969-4711-ae1e-6f6430114c5e None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:14.135298 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1617a87-8969-4711-ae1e-6f6430114c5e None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:14.135582 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1617a87-8969-4711-ae1e-6f6430114c5e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:14.137409 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1617a87-8969-4711-ae1e-6f6430114c5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:14.139049 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1617a87-8969-4711-ae1e-6f6430114c5e None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:14.140723 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 26/44] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Jul 20 22:25:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 22:25:14.146495 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1617a87-8969-4711-ae1e-6f6430114c5e None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:14.179196 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 19/45] 10.222.0.81 () {60 vars in 1131 bytes} [Wed Jul 20 22:25:13 2022] POST /identity/v3/roles => generated 223 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:25:14.205571 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:14.270020 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1617a87-8969-4711-ae1e-6f6430114c5e None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:14.307233 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee394cf-4d23-4ecb-b0ef-a8652cf4f772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:14.307919 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee394cf-4d23-4ecb-b0ef-a8652cf4f772 None admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:14.308209 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee394cf-4d23-4ecb-b0ef-a8652cf4f772 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:14.308494 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee394cf-4d23-4ecb-b0ef-a8652cf4f772 None admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:14.309775 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee394cf-4d23-4ecb-b0ef-a8652cf4f772 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:14.311301 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee394cf-4d23-4ecb-b0ef-a8652cf4f772 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:14.402566 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 27/46] 10.222.0.81 () {60 vars in 1141 bytes} [Wed Jul 20 22:25:14 2022] POST /identity/v3/projects => generated 312 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 22:25:15.719146 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a35a886-0084-45a1-8584-67c434fd9af9 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:15.719690 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a35a886-0084-45a1-8584-67c434fd9af9 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:15.719775 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a35a886-0084-45a1-8584-67c434fd9af9 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:15.721086 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 20/47] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:25:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:25:15.726452 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c3698f-d250-42f7-8399-ca494ab2d9d5 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:15.726865 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c3698f-d250-42f7-8399-ca494ab2d9d5 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:15.727052 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c3698f-d250-42f7-8399-ca494ab2d9d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:15.744024 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-79c3698f-d250-42f7-8399-ca494ab2d9d5 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:25:15.745207 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-79c3698f-d250-42f7-8399-ca494ab2d9d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:25:15.811665 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 28/48] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:25:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 22:25:15.834048 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a35a886-0084-45a1-8584-67c434fd9af9 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:15.939419 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a35a886-0084-45a1-8584-67c434fd9af9 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:15.968340 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:15.974257 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249667e9-85be-422d-9385-407f16fc22f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:15.975033 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249667e9-85be-422d-9385-407f16fc22f7 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:15.975341 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249667e9-85be-422d-9385-407f16fc22f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:15.975720 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249667e9-85be-422d-9385-407f16fc22f7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:15.987739 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249667e9-85be-422d-9385-407f16fc22f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 22:25:15.997087 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-249667e9-85be-422d-9385-407f16fc22f7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 22:25:15.997087 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:25:15.997087 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:25:15.997087 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:25:15.997087 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:25:15.997087 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:25:15.997087 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:25:15.997087 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:25:15.997087 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:25:15.997087 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:25:15.997087 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:25:15.997087 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:25:15.997087 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 22:25:15.997087 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 22:25:15.997087 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 22:25:15.997087 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 22:25:15.997994 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 22:25:15.997994 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 22:25:15.997994 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 22:25:15.997994 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 22:25:15.997994 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:25:15.997994 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:25:15.997994 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:25:15.997994 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:25:15.997994 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:25:15.997994 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:25:15.997994 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:25:15.997994 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:25:15.997994 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:25:15.997994 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:25:15.997994 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:25:15.997994 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:25:15.999370 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:25:15.999370 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:25:15.999370 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:25:15.999370 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:25:15.999370 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 22:25:15.999370 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 22:25:15.999370 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 22:25:15.999370 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 22:25:15.999370 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 22:25:15.999370 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 22:25:15.999370 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 22:25:15.999370 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:25:15.999370 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 21/49] 10.222.0.81 () {56 vars in 1096 bytes} [Wed Jul 20 22:25:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:25:16.004833 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c3698f-d250-42f7-8399-ca494ab2d9d5 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:16.029341 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:16.104408 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c3698f-d250-42f7-8399-ca494ab2d9d5 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:16.138662 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c4eb10-3381-4236-aa7c-2750d3f68130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:16.139429 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c4eb10-3381-4236-aa7c-2750d3f68130 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:16.139724 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c4eb10-3381-4236-aa7c-2750d3f68130 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:16.140016 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c4eb10-3381-4236-aa7c-2750d3f68130 None admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:16.140766 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c4eb10-3381-4236-aa7c-2750d3f68130 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:25:16.141079 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c4eb10-3381-4236-aa7c-2750d3f68130 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:16.142471 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c4eb10-3381-4236-aa7c-2750d3f68130 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:16.150587 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 29/50] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Jul 20 22:25:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:25:16.156311 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249667e9-85be-422d-9385-407f16fc22f7 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:16.255052 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249667e9-85be-422d-9385-407f16fc22f7 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:16.287353 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c617046-6d50-43f2-b37d-26c68f37426e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:16.288240 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c617046-6d50-43f2-b37d-26c68f37426e None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:16.288527 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c617046-6d50-43f2-b37d-26c68f37426e None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:16.288802 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c617046-6d50-43f2-b37d-26c68f37426e None admin] PATH_INFO: `/v3/users/98f5d4c4be4845478aa0d31a1403d775` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:16.289994 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c617046-6d50-43f2-b37d-26c68f37426e None admin] RBAC: Authorizing `identity:get_user(user_id=98f5d4c4be4845478aa0d31a1403d775)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:16.291511 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c617046-6d50-43f2-b37d-26c68f37426e None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:16.293815 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 22/51] 10.222.0.81 () {56 vars in 1177 bytes} [Wed Jul 20 22:25:16 2022] GET /identity/v3/users/98f5d4c4be4845478aa0d31a1403d775 => generated 247 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:25:16.299597 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c4eb10-3381-4236-aa7c-2750d3f68130 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:16.384239 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:16.400154 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c4eb10-3381-4236-aa7c-2750d3f68130 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:16.434525 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4955819-8619-41fc-93eb-137fb99a8eff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:16.435361 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4955819-8619-41fc-93eb-137fb99a8eff None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:16.435668 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4955819-8619-41fc-93eb-137fb99a8eff None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:16.435947 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4955819-8619-41fc-93eb-137fb99a8eff None admin] PATH_INFO: `/v3/domains/default` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:16.437118 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4955819-8619-41fc-93eb-137fb99a8eff None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:16.438558 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4955819-8619-41fc-93eb-137fb99a8eff None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:16.440198 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 30/52] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Jul 20 22:25:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:25:16.445898 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c617046-6d50-43f2-b37d-26c68f37426e None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:16.471789 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8a90ee-3526-4fc9-90e6-2fcab65c0fa4 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:16.472089 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8a90ee-3526-4fc9-90e6-2fcab65c0fa4 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:16.472374 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8a90ee-3526-4fc9-90e6-2fcab65c0fa4 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:16.473735 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 31/53] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:25:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 22:25:16.478740 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b8ab16-87d8-4b6c-8ebc-b8fd7028cb2a None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:16.479054 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b8ab16-87d8-4b6c-8ebc-b8fd7028cb2a None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:16.479338 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b8ab16-87d8-4b6c-8ebc-b8fd7028cb2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:16.496207 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-35b8ab16-87d8-4b6c-8ebc-b8fd7028cb2a None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:25:16.497426 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-35b8ab16-87d8-4b6c-8ebc-b8fd7028cb2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:25:16.505388 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:16.547803 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c617046-6d50-43f2-b37d-26c68f37426e None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:16.565852 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 32/54] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:25:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 22:25:16.580133 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6593f152-293c-40f3-bd1d-e4b514dd0c65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:16.580798 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6593f152-293c-40f3-bd1d-e4b514dd0c65 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:16.581088 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6593f152-293c-40f3-bd1d-e4b514dd0c65 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:16.581370 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6593f152-293c-40f3-bd1d-e4b514dd0c65 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:16.582204 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6593f152-293c-40f3-bd1d-e4b514dd0c65 None admin] RBAC: Adding query filter params (role.id=d89b6c92d35a401a8517f1d47f6be843, scope.domain.id=default, user.id=98f5d4c4be4845478aa0d31a1403d775) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:25:16.582519 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6593f152-293c-40f3-bd1d-e4b514dd0c65 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:16.584001 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6593f152-293c-40f3-bd1d-e4b514dd0c65 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:16.588636 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b8ab16-87d8-4b6c-8ebc-b8fd7028cb2a None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:16.592474 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 23/55] 10.222.0.81 () {56 vars in 1322 bytes} [Wed Jul 20 22:25:16 2022] GET /identity/v3/role_assignments?user.id=98f5d4c4be4845478aa0d31a1403d775&scope.domain.id=default&role.id=d89b6c92d35a401a8517f1d47f6be843 => generated 530 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:25:16.678703 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:16.697737 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b8ab16-87d8-4b6c-8ebc-b8fd7028cb2a None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:16.735097 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f2d61c-8948-4166-b456-513a653f0559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:16.735914 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f2d61c-8948-4166-b456-513a653f0559 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:16.736201 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f2d61c-8948-4166-b456-513a653f0559 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:16.736480 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f2d61c-8948-4166-b456-513a653f0559 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:16.737670 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f2d61c-8948-4166-b456-513a653f0559 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:16.739156 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f2d61c-8948-4166-b456-513a653f0559 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:16.740776 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 33/56] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Jul 20 22:25:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 22:25:16.747484 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6593f152-293c-40f3-bd1d-e4b514dd0c65 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:16.851404 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6593f152-293c-40f3-bd1d-e4b514dd0c65 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:16.873727 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:16.884762 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafbbc0e-f066-428f-8105-907813fd66b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:16.885488 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafbbc0e-f066-428f-8105-907813fd66b1 None admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:16.885788 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafbbc0e-f066-428f-8105-907813fd66b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:16.886073 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafbbc0e-f066-428f-8105-907813fd66b1 None admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:16.886818 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafbbc0e-f066-428f-8105-907813fd66b1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:16.888830 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafbbc0e-f066-428f-8105-907813fd66b1 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:16.977613 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 24/57] 10.222.0.81 () {60 vars in 1140 bytes} [Wed Jul 20 22:25:16 2022] POST /identity/v3/projects => generated 298 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:25:19.036388 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091fb935-c9c9-4270-8da4-04a52a0d90ea None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:19.036910 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091fb935-c9c9-4270-8da4-04a52a0d90ea None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:19.036993 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091fb935-c9c9-4270-8da4-04a52a0d90ea None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:19.038320 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 34/58] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:25:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:25:19.044117 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b65200-4fb9-4f28-98ca-759bec94abfd None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:19.044489 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b65200-4fb9-4f28-98ca-759bec94abfd None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:19.044677 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b65200-4fb9-4f28-98ca-759bec94abfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:19.061310 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-15b65200-4fb9-4f28-98ca-759bec94abfd None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:25:19.062515 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-15b65200-4fb9-4f28-98ca-759bec94abfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:25:19.231751 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 25/59] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:25:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 22:25:19.256388 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091fb935-c9c9-4270-8da4-04a52a0d90ea None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:19.356101 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091fb935-c9c9-4270-8da4-04a52a0d90ea None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:19.389508 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fabf1e-7561-496f-8ed9-81c808a86a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:19.390301 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fabf1e-7561-496f-8ed9-81c808a86a69 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:19.390595 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fabf1e-7561-496f-8ed9-81c808a86a69 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:19.390878 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fabf1e-7561-496f-8ed9-81c808a86a69 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:19.392104 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fabf1e-7561-496f-8ed9-81c808a86a69 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:19.393551 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fabf1e-7561-496f-8ed9-81c808a86a69 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:19.395211 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 35/60] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Jul 20 22:25:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:25:19.400835 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b65200-4fb9-4f28-98ca-759bec94abfd None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:19.495899 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b65200-4fb9-4f28-98ca-759bec94abfd None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:19.498131 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:19.527946 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7796444-8a9d-4cf6-9e80-c51bbd6ff838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:19.528614 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7796444-8a9d-4cf6-9e80-c51bbd6ff838 None admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:19.528905 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7796444-8a9d-4cf6-9e80-c51bbd6ff838 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:19.529185 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7796444-8a9d-4cf6-9e80-c51bbd6ff838 None admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:19.529894 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7796444-8a9d-4cf6-9e80-c51bbd6ff838 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:19.531357 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7796444-8a9d-4cf6-9e80-c51bbd6ff838 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:19.674320 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 26/61] 10.222.0.81 () {60 vars in 1132 bytes} [Wed Jul 20 22:25:19 2022] POST /identity/v3/users => generated 275 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:25:22.699735 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38571150-d71d-4520-8fa9-b459c12b3e6e None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:22.700780 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38571150-d71d-4520-8fa9-b459c12b3e6e None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:22.701378 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38571150-d71d-4520-8fa9-b459c12b3e6e None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:22.702958 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 36/62] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:25:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:25:22.708351 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e46695-a3ae-4f26-829c-954d20948e5c None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:22.709173 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e46695-a3ae-4f26-829c-954d20948e5c None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:22.709655 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e46695-a3ae-4f26-829c-954d20948e5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:22.735952 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-22e46695-a3ae-4f26-829c-954d20948e5c None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:25:22.737478 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-22e46695-a3ae-4f26-829c-954d20948e5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:25:22.825528 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 27/63] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:25:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 22:25:22.861027 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38571150-d71d-4520-8fa9-b459c12b3e6e None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:22.891774 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:22.981229 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c0d38c-c8b0-4c96-9104-6dee17a9066b None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:22.981534 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c0d38c-c8b0-4c96-9104-6dee17a9066b None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:22.981729 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c0d38c-c8b0-4c96-9104-6dee17a9066b None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:22.983416 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 28/64] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:25:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:25:23.000998 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ff248c-8d23-4844-a160-49940f15aeb8 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:23.001296 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ff248c-8d23-4844-a160-49940f15aeb8 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:23.001588 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ff248c-8d23-4844-a160-49940f15aeb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:23.008946 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38571150-d71d-4520-8fa9-b459c12b3e6e None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:23.027559 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-d4ff248c-8d23-4844-a160-49940f15aeb8 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:25:23.029730 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-d4ff248c-8d23-4844-a160-49940f15aeb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:25:23.070445 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd65cb53-8add-4c37-89a7-bfff2a935284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:23.071654 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd65cb53-8add-4c37-89a7-bfff2a935284 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:23.072142 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd65cb53-8add-4c37-89a7-bfff2a935284 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:23.072615 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd65cb53-8add-4c37-89a7-bfff2a935284 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:23.093851 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd65cb53-8add-4c37-89a7-bfff2a935284 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 22:25:23.111711 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-bd65cb53-8add-4c37-89a7-bfff2a935284 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 22:25:23.111711 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:25:23.111711 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:25:23.111711 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:25:23.111711 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:25:23.111711 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:25:23.111711 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:25:23.111711 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:25:23.111711 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:25:23.111711 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:25:23.111711 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:25:23.111711 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:25:23.111711 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 22:25:23.111711 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 22:25:23.111711 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 22:25:23.111711 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 22:25:23.112625 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 22:25:23.112625 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 22:25:23.112625 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 22:25:23.112625 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 22:25:23.112625 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:25:23.112625 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:25:23.112625 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:25:23.112625 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:25:23.112625 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:25:23.112625 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:25:23.112625 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:25:23.112625 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:25:23.112625 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:25:23.112625 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:25:23.112625 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:25:23.112625 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:25:23.113383 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:25:23.113383 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:25:23.113383 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:25:23.113383 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:25:23.113383 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 22:25:23.113383 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 22:25:23.113383 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 22:25:23.113383 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 22:25:23.113383 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 22:25:23.113383 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 22:25:23.113383 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 22:25:23.113383 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:25:23.113383 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 37/65] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Jul 20 22:25:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:25:23.122879 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd65cb53-8add-4c37-89a7-bfff2a935284 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:23.165813 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 29/66] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:25:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 22:25:23.205306 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ff248c-8d23-4844-a160-49940f15aeb8 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:23.235806 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:23.266645 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:23.301024 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd65cb53-8add-4c37-89a7-bfff2a935284 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:23.338193 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b1190f-9ff3-450e-aaa9-4b5d926e7126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:23.338878 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b1190f-9ff3-450e-aaa9-4b5d926e7126 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:23.339743 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b1190f-9ff3-450e-aaa9-4b5d926e7126 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:23.340034 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b1190f-9ff3-450e-aaa9-4b5d926e7126 None admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:23.340796 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b1190f-9ff3-450e-aaa9-4b5d926e7126 None admin] RBAC: Adding query filter params (name=member) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:25:23.341113 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b1190f-9ff3-450e-aaa9-4b5d926e7126 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:23.342628 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b1190f-9ff3-450e-aaa9-4b5d926e7126 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:23.351779 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 38/67] 10.222.0.81 () {56 vars in 1101 bytes} [Wed Jul 20 22:25:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 22:25:23.357415 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e2c428-df44-441e-8aac-c90d5171097d None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:23.357415 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e2c428-df44-441e-8aac-c90d5171097d None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:23.357415 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e2c428-df44-441e-8aac-c90d5171097d None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:23.357415 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 39/68] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:25:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:25:23.358567 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e2c428-df44-441e-8aac-c90d5171097d None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:23.360425 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ff248c-8d23-4844-a160-49940f15aeb8 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:23.397930 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5086e4-3d1a-4e8b-a74f-5c0e281ea830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:23.398988 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5086e4-3d1a-4e8b-a74f-5c0e281ea830 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:23.399454 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5086e4-3d1a-4e8b-a74f-5c0e281ea830 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:23.399875 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5086e4-3d1a-4e8b-a74f-5c0e281ea830 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:23.412345 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5086e4-3d1a-4e8b-a74f-5c0e281ea830 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 22:25:23.421665 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-6d5086e4-3d1a-4e8b-a74f-5c0e281ea830 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 22:25:23.421665 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:25:23.421665 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:25:23.421665 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:25:23.421665 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:25:23.421665 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:25:23.421665 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:25:23.421665 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:25:23.421665 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:25:23.421665 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:25:23.421665 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:25:23.421665 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:25:23.421665 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 22:25:23.421665 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 22:25:23.421665 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 22:25:23.421665 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 22:25:23.422492 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 22:25:23.422492 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 22:25:23.422492 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 22:25:23.422492 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 22:25:23.422492 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:25:23.422492 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:25:23.422492 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:25:23.422492 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:25:23.422492 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:25:23.422492 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:25:23.422492 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:25:23.422492 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:25:23.422492 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:25:23.422492 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:25:23.422492 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:25:23.422492 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:25:23.423274 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:25:23.423274 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:25:23.423274 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:25:23.423274 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:25:23.423274 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 22:25:23.423274 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 22:25:23.423274 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 22:25:23.423274 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 22:25:23.423274 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 22:25:23.423274 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 22:25:23.423274 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 22:25:23.423274 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:25:23.425712 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 30/69] 10.222.0.81 () {56 vars in 1096 bytes} [Wed Jul 20 22:25:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 22:25:23.427541 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e46aa74-01d6-4c56-8c0c-099e71c5e1f0 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:23.428015 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e46aa74-01d6-4c56-8c0c-099e71c5e1f0 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:23.428484 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e46aa74-01d6-4c56-8c0c-099e71c5e1f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:23.446317 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-9e46aa74-01d6-4c56-8c0c-099e71c5e1f0 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:25:23.447945 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-9e46aa74-01d6-4c56-8c0c-099e71c5e1f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:25:23.483279 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e2c428-df44-441e-8aac-c90d5171097d None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:23.523627 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 31/70] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:25:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 22:25:23.524911 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e46aa74-01d6-4c56-8c0c-099e71c5e1f0 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:23.534989 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4661429b-8e95-4e0e-8174-033d72930066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:23.535936 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4661429b-8e95-4e0e-8174-033d72930066 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:23.536224 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4661429b-8e95-4e0e-8174-033d72930066 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:23.536509 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4661429b-8e95-4e0e-8174-033d72930066 None admin] PATH_INFO: `/v3/users/74bf4b54c7664f9fac20de5b75d465f8` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:23.541671 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:23.550446 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:23.554064 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4661429b-8e95-4e0e-8174-033d72930066 None admin] RBAC: Authorizing `identity:get_user(user_id=74bf4b54c7664f9fac20de5b75d465f8)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:23.555577 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4661429b-8e95-4e0e-8174-033d72930066 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:23.557464 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 40/71] 10.222.0.81 () {56 vars in 1177 bytes} [Wed Jul 20 22:25:23 2022] GET /identity/v3/users/74bf4b54c7664f9fac20de5b75d465f8 => generated 275 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:25:23.559199 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecb1f64-3f7e-43d3-9428-f63ade0667e7 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:23.559491 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecb1f64-3f7e-43d3-9428-f63ade0667e7 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:23.559774 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecb1f64-3f7e-43d3-9428-f63ade0667e7 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:23.561160 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 41/72] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:25:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:25:23.564311 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26245c57-14ac-4984-8642-6c5f181a90d7 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:23.564614 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26245c57-14ac-4984-8642-6c5f181a90d7 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:23.565219 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26245c57-14ac-4984-8642-6c5f181a90d7 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:23.566504 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 42/73] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:25:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:25:23.567307 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26245c57-14ac-4984-8642-6c5f181a90d7 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:23.633336 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e46aa74-01d6-4c56-8c0c-099e71c5e1f0 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:23.665914 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9042bf-f32c-494a-9709-2ce00a275c18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:23.666590 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9042bf-f32c-494a-9709-2ce00a275c18 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:23.666877 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9042bf-f32c-494a-9709-2ce00a275c18 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:23.667684 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26245c57-14ac-4984-8642-6c5f181a90d7 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:23.667791 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9042bf-f32c-494a-9709-2ce00a275c18 None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:23.668464 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9042bf-f32c-494a-9709-2ce00a275c18 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:25:23.668773 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9042bf-f32c-494a-9709-2ce00a275c18 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:23.670174 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9042bf-f32c-494a-9709-2ce00a275c18 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:23.678538 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 32/74] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Jul 20 22:25:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:25:23.679377 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9042bf-f32c-494a-9709-2ce00a275c18 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:23.702681 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4505ddcb-b30f-495b-b770-68984a405297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:23.703545 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4505ddcb-b30f-495b-b770-68984a405297 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:23.703838 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4505ddcb-b30f-495b-b770-68984a405297 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:23.704123 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4505ddcb-b30f-495b-b770-68984a405297 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:23.714099 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4505ddcb-b30f-495b-b770-68984a405297 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 22:25:23.722659 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-4505ddcb-b30f-495b-b770-68984a405297 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 22:25:23.722659 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:25:23.722659 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:25:23.722659 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:25:23.722659 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:25:23.722659 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:25:23.722659 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:25:23.722659 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:25:23.722659 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:25:23.722659 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:25:23.722659 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:25:23.722659 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:25:23.722659 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 22:25:23.722659 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 22:25:23.722659 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 22:25:23.722659 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 22:25:23.723577 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 22:25:23.723577 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 22:25:23.723577 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 22:25:23.723577 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 22:25:23.723577 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:25:23.723577 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:25:23.723577 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:25:23.723577 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:25:23.723577 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:25:23.723577 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:25:23.723577 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:25:23.723577 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:25:23.723577 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:25:23.723577 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:25:23.723577 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:25:23.723577 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:25:23.724326 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:25:23.724326 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:25:23.724326 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:25:23.724326 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:25:23.724326 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 22:25:23.724326 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 22:25:23.724326 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 22:25:23.724326 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 22:25:23.724326 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 22:25:23.724326 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 22:25:23.724326 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 22:25:23.724326 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:25:23.724326 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 43/75] 10.222.0.81 () {56 vars in 1114 bytes} [Wed Jul 20 22:25:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:25:23.725190 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3af2f28-e056-4ae6-a1db-e1c79b3f3a3a None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:23.725370 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3af2f28-e056-4ae6-a1db-e1c79b3f3a3a None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:23.725656 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3af2f28-e056-4ae6-a1db-e1c79b3f3a3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:23.742101 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-d3af2f28-e056-4ae6-a1db-e1c79b3f3a3a None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:25:23.743347 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-d3af2f28-e056-4ae6-a1db-e1c79b3f3a3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:25:23.752735 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:23.781725 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9042bf-f32c-494a-9709-2ce00a275c18 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:23.798476 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:23.811944 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 44/76] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:25:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 22:25:23.812970 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9ac7ed-a7ad-492f-a3bf-e51b69a252cd None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:23.813263 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9ac7ed-a7ad-492f-a3bf-e51b69a252cd None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:23.813553 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9ac7ed-a7ad-492f-a3bf-e51b69a252cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:23.815826 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4a9c8a-8fad-4582-b240-cd22f8b22720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:23.816489 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4a9c8a-8fad-4582-b240-cd22f8b22720 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:23.816776 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4a9c8a-8fad-4582-b240-cd22f8b22720 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:23.817054 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4a9c8a-8fad-4582-b240-cd22f8b22720 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:23.829160 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-3a9ac7ed-a7ad-492f-a3bf-e51b69a252cd None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:25:23.830820 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-3a9ac7ed-a7ad-492f-a3bf-e51b69a252cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:25:23.894095 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4a9c8a-8fad-4582-b240-cd22f8b22720 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:23.895615 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4a9c8a-8fad-4582-b240-cd22f8b22720 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:23.900918 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 45/77] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:25:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 22:25:23.901773 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9ac7ed-a7ad-492f-a3bf-e51b69a252cd None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:23.963502 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:23.999003 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 33/78] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Jul 20 22:25:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 22:25:23.999003 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4a9c8a-8fad-4582-b240-cd22f8b22720 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:24.021484 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9ac7ed-a7ad-492f-a3bf-e51b69a252cd None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:24.061538 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:24.061538 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbe1482-d64d-4f2a-9281-d46eb852e672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:24.061538 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbe1482-d64d-4f2a-9281-d46eb852e672 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:24.061538 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbe1482-d64d-4f2a-9281-d46eb852e672 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:24.061998 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbe1482-d64d-4f2a-9281-d46eb852e672 None admin] PATH_INFO: `/v3/users/98f5d4c4be4845478aa0d31a1403d775` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:24.061998 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbe1482-d64d-4f2a-9281-d46eb852e672 None admin] RBAC: Authorizing `identity:get_user(user_id=98f5d4c4be4845478aa0d31a1403d775)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:24.061998 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbe1482-d64d-4f2a-9281-d46eb852e672 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:24.062503 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 46/79] 10.222.0.81 () {56 vars in 1177 bytes} [Wed Jul 20 22:25:23 2022] GET /identity/v3/users/98f5d4c4be4845478aa0d31a1403d775 => generated 247 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:25:24.063411 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbe1482-d64d-4f2a-9281-d46eb852e672 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:24.095537 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4a9c8a-8fad-4582-b240-cd22f8b22720 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:24.128819 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576e2c7b-374d-4a5e-8c1c-ce7898efbf9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:24.129498 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576e2c7b-374d-4a5e-8c1c-ce7898efbf9f None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:24.129787 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576e2c7b-374d-4a5e-8c1c-ce7898efbf9f None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:24.130127 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576e2c7b-374d-4a5e-8c1c-ce7898efbf9f None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:24.130868 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576e2c7b-374d-4a5e-8c1c-ce7898efbf9f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:25:24.131256 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576e2c7b-374d-4a5e-8c1c-ce7898efbf9f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:24.132648 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576e2c7b-374d-4a5e-8c1c-ce7898efbf9f None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:24.140974 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 34/80] 10.222.0.81 () {56 vars in 1111 bytes} [Wed Jul 20 22:25:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:25:24.141830 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576e2c7b-374d-4a5e-8c1c-ce7898efbf9f None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:24.170356 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbe1482-d64d-4f2a-9281-d46eb852e672 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:24.175938 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:24.205056 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae3853c-9d21-41f7-8039-cb4af8a99c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:24.205871 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae3853c-9d21-41f7-8039-cb4af8a99c55 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:24.206159 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae3853c-9d21-41f7-8039-cb4af8a99c55 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:24.206439 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae3853c-9d21-41f7-8039-cb4af8a99c55 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:24.217062 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae3853c-9d21-41f7-8039-cb4af8a99c55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 22:25:24.225128 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-bae3853c-9d21-41f7-8039-cb4af8a99c55 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 22:25:24.225128 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:25:24.225128 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:25:24.225128 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:25:24.225128 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:25:24.225128 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:25:24.225128 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:25:24.225128 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:25:24.225128 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:25:24.225128 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:25:24.225128 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:25:24.225128 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:25:24.225128 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 22:25:24.225128 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 22:25:24.225128 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 22:25:24.225128 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 22:25:24.226001 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 22:25:24.226001 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 22:25:24.226001 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 22:25:24.226001 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 22:25:24.226001 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:25:24.226001 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:25:24.226001 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:25:24.226001 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:25:24.226001 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:25:24.226001 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:25:24.226001 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:25:24.226001 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:25:24.226001 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:25:24.226001 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:25:24.226001 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:25:24.226001 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:25:24.226773 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:25:24.226773 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:25:24.226773 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:25:24.226773 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:25:24.226773 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 22:25:24.226773 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 22:25:24.226773 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 22:25:24.226773 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 22:25:24.226773 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 22:25:24.226773 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 22:25:24.226773 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 22:25:24.226773 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:25:24.226773 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 47/81] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Jul 20 22:25:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:25:24.227518 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae3853c-9d21-41f7-8039-cb4af8a99c55 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:24.243613 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576e2c7b-374d-4a5e-8c1c-ce7898efbf9f None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:24.278002 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777b4b61-a0af-4c3e-95dd-9d0f7d050d15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:24.278546 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:24.278823 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777b4b61-a0af-4c3e-95dd-9d0f7d050d15 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:24.279150 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777b4b61-a0af-4c3e-95dd-9d0f7d050d15 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:24.279436 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777b4b61-a0af-4c3e-95dd-9d0f7d050d15 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:24.280643 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777b4b61-a0af-4c3e-95dd-9d0f7d050d15 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:24.282108 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777b4b61-a0af-4c3e-95dd-9d0f7d050d15 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:24.283776 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 35/82] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Jul 20 22:25:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:25:24.284549 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777b4b61-a0af-4c3e-95dd-9d0f7d050d15 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:24.332766 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae3853c-9d21-41f7-8039-cb4af8a99c55 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:24.366276 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63baab53-1139-4d4e-8136-7a93cdd61187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:24.367194 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63baab53-1139-4d4e-8136-7a93cdd61187 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:24.367482 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63baab53-1139-4d4e-8136-7a93cdd61187 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:24.367768 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63baab53-1139-4d4e-8136-7a93cdd61187 None admin] PATH_INFO: `/v3/projects/da54f6860e1f43c29cf90271280bdcae` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:24.368961 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63baab53-1139-4d4e-8136-7a93cdd61187 None admin] RBAC: Authorizing `identity:get_project(project_id=da54f6860e1f43c29cf90271280bdcae)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:24.370426 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63baab53-1139-4d4e-8136-7a93cdd61187 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:24.372171 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 48/83] 10.222.0.81 () {56 vars in 1186 bytes} [Wed Jul 20 22:25:24 2022] GET /identity/v3/projects/da54f6860e1f43c29cf90271280bdcae => generated 298 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:25:24.372933 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63baab53-1139-4d4e-8136-7a93cdd61187 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:24.384959 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777b4b61-a0af-4c3e-95dd-9d0f7d050d15 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:24.387026 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:24.419630 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e3f7cf-d0f3-454e-b367-6c1798d1cee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:24.420302 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e3f7cf-d0f3-454e-b367-6c1798d1cee0 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:24.420842 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e3f7cf-d0f3-454e-b367-6c1798d1cee0 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:24.421124 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e3f7cf-d0f3-454e-b367-6c1798d1cee0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:24.477062 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63baab53-1139-4d4e-8136-7a93cdd61187 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:24.489235 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e3f7cf-d0f3-454e-b367-6c1798d1cee0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:24.490159 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:24.490972 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e3f7cf-d0f3-454e-b367-6c1798d1cee0 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:24.509885 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f88144e-14c3-4d57-b2c1-dc5058858852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:24.510770 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f88144e-14c3-4d57-b2c1-dc5058858852 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:24.511698 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f88144e-14c3-4d57-b2c1-dc5058858852 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:24.512019 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f88144e-14c3-4d57-b2c1-dc5058858852 None admin] PATH_INFO: `/v3/users/74bf4b54c7664f9fac20de5b75d465f8` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:24.513221 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f88144e-14c3-4d57-b2c1-dc5058858852 None admin] RBAC: Authorizing `identity:get_user(user_id=74bf4b54c7664f9fac20de5b75d465f8)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:24.514656 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f88144e-14c3-4d57-b2c1-dc5058858852 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:24.516336 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 49/84] 10.222.0.81 () {56 vars in 1177 bytes} [Wed Jul 20 22:25:24 2022] GET /identity/v3/users/74bf4b54c7664f9fac20de5b75d465f8 => generated 275 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:25:24.517100 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f88144e-14c3-4d57-b2c1-dc5058858852 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:24.590260 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 36/85] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Jul 20 22:25:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 22:25:24.591121 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e3f7cf-d0f3-454e-b367-6c1798d1cee0 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:24.593485 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:24.617234 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f88144e-14c3-4d57-b2c1-dc5058858852 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:24.650348 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70046af-4f9c-410e-9bbc-54366fc1ac7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:24.651040 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70046af-4f9c-410e-9bbc-54366fc1ac7e None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:24.651341 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70046af-4f9c-410e-9bbc-54366fc1ac7e None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:24.652161 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70046af-4f9c-410e-9bbc-54366fc1ac7e None admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:24.652920 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70046af-4f9c-410e-9bbc-54366fc1ac7e None admin] RBAC: Adding query filter params (name=member) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:25:24.653237 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70046af-4f9c-410e-9bbc-54366fc1ac7e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:24.654632 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70046af-4f9c-410e-9bbc-54366fc1ac7e None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:24.663265 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 50/86] 10.222.0.81 () {56 vars in 1101 bytes} [Wed Jul 20 22:25:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:25:24.664226 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70046af-4f9c-410e-9bbc-54366fc1ac7e None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:24.688365 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e3f7cf-d0f3-454e-b367-6c1798d1cee0 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:24.710113 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:24.721392 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab5c65a-74d5-4078-95e7-f14b30ab0549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:24.722069 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab5c65a-74d5-4078-95e7-f14b30ab0549 None admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:24.722357 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab5c65a-74d5-4078-95e7-f14b30ab0549 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:24.722634 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab5c65a-74d5-4078-95e7-f14b30ab0549 None admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:24.723377 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab5c65a-74d5-4078-95e7-f14b30ab0549 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:24.724821 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab5c65a-74d5-4078-95e7-f14b30ab0549 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:24.763162 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70046af-4f9c-410e-9bbc-54366fc1ac7e None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:24.796674 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509148eb-9f6b-41a8-80d7-322dfc5aa664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:24.797321 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509148eb-9f6b-41a8-80d7-322dfc5aa664 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:24.797614 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509148eb-9f6b-41a8-80d7-322dfc5aa664 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:24.797896 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509148eb-9f6b-41a8-80d7-322dfc5aa664 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:24.798723 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509148eb-9f6b-41a8-80d7-322dfc5aa664 None admin] RBAC: Adding query filter params (role.id=559942c4de1744169f8b5e0404ecde6b, scope.project.id=da54f6860e1f43c29cf90271280bdcae, user.id=74bf4b54c7664f9fac20de5b75d465f8) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:25:24.799064 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509148eb-9f6b-41a8-80d7-322dfc5aa664 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:24.800496 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509148eb-9f6b-41a8-80d7-322dfc5aa664 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:24.805829 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 51/87] 10.222.0.81 () {56 vars in 1374 bytes} [Wed Jul 20 22:25:24 2022] GET /identity/v3/role_assignments?user.id=74bf4b54c7664f9fac20de5b75d465f8&scope.project.id=da54f6860e1f43c29cf90271280bdcae&role.id=559942c4de1744169f8b5e0404ecde6b => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:25:24.806597 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509148eb-9f6b-41a8-80d7-322dfc5aa664 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:24.916977 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509148eb-9f6b-41a8-80d7-322dfc5aa664 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:24.927645 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:24.959038 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 37/88] 10.222.0.81 () {60 vars in 1132 bytes} [Wed Jul 20 22:25:24 2022] POST /identity/v3/users => generated 289 bytes in 358 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:25:24.959038 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab5c65a-74d5-4078-95e7-f14b30ab0549 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:24.965657 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1832e6-22d1-479e-a942-4bd94ff27ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:24.966336 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1832e6-22d1-479e-a942-4bd94ff27ebf None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:24.966634 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1832e6-22d1-479e-a942-4bd94ff27ebf None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:24.966944 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1832e6-22d1-479e-a942-4bd94ff27ebf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:25.037096 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:25.068529 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1832e6-22d1-479e-a942-4bd94ff27ebf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:25.070389 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1832e6-22d1-479e-a942-4bd94ff27ebf None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:25.086404 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab5c65a-74d5-4078-95e7-f14b30ab0549 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:25.130425 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddc5054-d4ec-4c2e-bf73-8e5f827201a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:25.131491 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddc5054-d4ec-4c2e-bf73-8e5f827201a5 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:25.131692 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddc5054-d4ec-4c2e-bf73-8e5f827201a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:25.131979 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddc5054-d4ec-4c2e-bf73-8e5f827201a5 None admin] PATH_INFO: `/v3/projects/da54f6860e1f43c29cf90271280bdcae` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:25.133634 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddc5054-d4ec-4c2e-bf73-8e5f827201a5 None admin] RBAC: Authorizing `identity:get_project(project_id=da54f6860e1f43c29cf90271280bdcae)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:25.134652 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddc5054-d4ec-4c2e-bf73-8e5f827201a5 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:25.141339 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 38/89] 10.222.0.81 () {56 vars in 1186 bytes} [Wed Jul 20 22:25:24 2022] GET /identity/v3/projects/da54f6860e1f43c29cf90271280bdcae => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:25:25.142134 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddc5054-d4ec-4c2e-bf73-8e5f827201a5 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:25.202096 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 52/90] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Jul 20 22:25:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 22:25:25.203103 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1832e6-22d1-479e-a942-4bd94ff27ebf None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:25.216051 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:25.292215 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddc5054-d4ec-4c2e-bf73-8e5f827201a5 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:25.328045 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:25.337682 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed2ce0c-1aa8-44ac-a6b3-a0ce9e7ffd77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:25.344211 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1832e6-22d1-479e-a942-4bd94ff27ebf None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:25.344332 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed2ce0c-1aa8-44ac-a6b3-a0ce9e7ffd77 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:25.344332 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed2ce0c-1aa8-44ac-a6b3-a0ce9e7ffd77 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:25.344332 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed2ce0c-1aa8-44ac-a6b3-a0ce9e7ffd77 None admin] PATH_INFO: `/v3/users/74bf4b54c7664f9fac20de5b75d465f8` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:25.344332 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed2ce0c-1aa8-44ac-a6b3-a0ce9e7ffd77 None admin] RBAC: Authorizing `identity:get_user(user_id=74bf4b54c7664f9fac20de5b75d465f8)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:25.344332 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed2ce0c-1aa8-44ac-a6b3-a0ce9e7ffd77 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:25.345338 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 39/91] 10.222.0.81 () {56 vars in 1177 bytes} [Wed Jul 20 22:25:25 2022] GET /identity/v3/users/74bf4b54c7664f9fac20de5b75d465f8 => generated 275 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:25:25.346324 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed2ce0c-1aa8-44ac-a6b3-a0ce9e7ffd77 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:25.386485 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25be9785-fc18-4547-ab85-fc7becd85f01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:25.387210 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25be9785-fc18-4547-ab85-fc7becd85f01 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:25.387512 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25be9785-fc18-4547-ab85-fc7becd85f01 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:25.387797 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25be9785-fc18-4547-ab85-fc7becd85f01 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:25.388655 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25be9785-fc18-4547-ab85-fc7becd85f01 None admin] RBAC: Adding query filter params (role.id=d89b6c92d35a401a8517f1d47f6be843, scope.project.id=da54f6860e1f43c29cf90271280bdcae, user.id=98f5d4c4be4845478aa0d31a1403d775) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:25:25.388979 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25be9785-fc18-4547-ab85-fc7becd85f01 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:25.390456 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25be9785-fc18-4547-ab85-fc7becd85f01 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:25.404082 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 53/92] 10.222.0.81 () {56 vars in 1374 bytes} [Wed Jul 20 22:25:25 2022] GET /identity/v3/role_assignments?user.id=98f5d4c4be4845478aa0d31a1403d775&scope.project.id=da54f6860e1f43c29cf90271280bdcae&role.id=d89b6c92d35a401a8517f1d47f6be843 => generated 259 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:25:25.404922 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25be9785-fc18-4547-ab85-fc7becd85f01 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:25.532464 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed2ce0c-1aa8-44ac-a6b3-a0ce9e7ffd77 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:25.550945 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:25.602418 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679e20e9-813d-4d2d-a039-7e1c817fc649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:25.603374 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679e20e9-813d-4d2d-a039-7e1c817fc649 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:25.603667 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679e20e9-813d-4d2d-a039-7e1c817fc649 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:25.603952 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679e20e9-813d-4d2d-a039-7e1c817fc649 None admin] PATH_INFO: `/v3/projects/da54f6860e1f43c29cf90271280bdcae` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:25.605828 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679e20e9-813d-4d2d-a039-7e1c817fc649 None admin] RBAC: Authorizing `identity:get_project(project_id=da54f6860e1f43c29cf90271280bdcae)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:25.607337 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679e20e9-813d-4d2d-a039-7e1c817fc649 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:25.609326 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 40/93] 10.222.0.81 () {56 vars in 1186 bytes} [Wed Jul 20 22:25:25 2022] GET /identity/v3/projects/da54f6860e1f43c29cf90271280bdcae => generated 298 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:25:25.611463 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25be9785-fc18-4547-ab85-fc7becd85f01 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:25.615563 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679e20e9-813d-4d2d-a039-7e1c817fc649 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:25.685235 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6defcf35-fbd4-401e-ae9b-a6fdcc1cde86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:25.685913 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6defcf35-fbd4-401e-ae9b-a6fdcc1cde86 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:25.686206 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6defcf35-fbd4-401e-ae9b-a6fdcc1cde86 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:25.686493 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6defcf35-fbd4-401e-ae9b-a6fdcc1cde86 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:25.740732 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:25.787120 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6defcf35-fbd4-401e-ae9b-a6fdcc1cde86 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:25.789576 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6defcf35-fbd4-401e-ae9b-a6fdcc1cde86 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:25.853186 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679e20e9-813d-4d2d-a039-7e1c817fc649 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:25.908099 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:25.940292 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e857b337-51c7-4f12-8bc4-9b931b099c57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:25.941032 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e857b337-51c7-4f12-8bc4-9b931b099c57 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:25.941337 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e857b337-51c7-4f12-8bc4-9b931b099c57 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:25.941634 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e857b337-51c7-4f12-8bc4-9b931b099c57 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:25.942474 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e857b337-51c7-4f12-8bc4-9b931b099c57 None admin] RBAC: Adding query filter params (role.id=6d5a3e6a22fa4f4fa47afcfce0246a1f, scope.project.id=da54f6860e1f43c29cf90271280bdcae, user.id=74bf4b54c7664f9fac20de5b75d465f8) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:25:25.942798 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e857b337-51c7-4f12-8bc4-9b931b099c57 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:25.944313 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e857b337-51c7-4f12-8bc4-9b931b099c57 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:25.949727 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 54/94] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Jul 20 22:25:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 545 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 22:25:25.955919 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6defcf35-fbd4-401e-ae9b-a6fdcc1cde86 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:25.957517 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 41/95] 10.222.0.81 () {56 vars in 1374 bytes} [Wed Jul 20 22:25:25 2022] GET /identity/v3/role_assignments?user.id=74bf4b54c7664f9fac20de5b75d465f8&scope.project.id=da54f6860e1f43c29cf90271280bdcae&role.id=6d5a3e6a22fa4f4fa47afcfce0246a1f => generated 259 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 22:25:26.100923 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6defcf35-fbd4-401e-ae9b-a6fdcc1cde86 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:26.151074 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1394434c-e1ec-4146-946b-164c513657ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:26.152782 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1394434c-e1ec-4146-946b-164c513657ad None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:26.153224 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1394434c-e1ec-4146-946b-164c513657ad None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:26.153652 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1394434c-e1ec-4146-946b-164c513657ad None admin] PATH_INFO: `/v3/projects/311b0d7cef0c483396ad292837f6d314` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:26.159872 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1394434c-e1ec-4146-946b-164c513657ad None admin] RBAC: Authorizing `identity:get_project(project_id=311b0d7cef0c483396ad292837f6d314)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:26.161761 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1394434c-e1ec-4146-946b-164c513657ad None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:26.163884 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 55/96] 10.222.0.81 () {56 vars in 1186 bytes} [Wed Jul 20 22:25:25 2022] GET /identity/v3/projects/311b0d7cef0c483396ad292837f6d314 => generated 312 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 22:25:26.182052 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e857b337-51c7-4f12-8bc4-9b931b099c57 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:26.321865 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e857b337-51c7-4f12-8bc4-9b931b099c57 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:26.352683 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:26.370688 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fff7555-2267-4f92-b6eb-675704210f31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:26.371679 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fff7555-2267-4f92-b6eb-675704210f31 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:26.372116 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fff7555-2267-4f92-b6eb-675704210f31 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:26.372545 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fff7555-2267-4f92-b6eb-675704210f31 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:26.373565 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fff7555-2267-4f92-b6eb-675704210f31 None admin] RBAC: Adding query filter params (role.id=559942c4de1744169f8b5e0404ecde6b, scope.project.id=311b0d7cef0c483396ad292837f6d314, user.id=74bf4b54c7664f9fac20de5b75d465f8) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:25:26.374049 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fff7555-2267-4f92-b6eb-675704210f31 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:26.375719 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fff7555-2267-4f92-b6eb-675704210f31 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:26.382295 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 42/97] 10.222.0.81 () {56 vars in 1374 bytes} [Wed Jul 20 22:25:26 2022] GET /identity/v3/role_assignments?user.id=74bf4b54c7664f9fac20de5b75d465f8&scope.project.id=311b0d7cef0c483396ad292837f6d314&role.id=559942c4de1744169f8b5e0404ecde6b => generated 259 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:25:27.730321 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3b74aa-4fdb-4ee1-b026-c162b37507a8 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:27.731376 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3b74aa-4fdb-4ee1-b026-c162b37507a8 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:27.731894 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3b74aa-4fdb-4ee1-b026-c162b37507a8 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:27.733503 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 56/98] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:25:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:25:27.738965 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86493401-95fb-444c-a905-7f31902f771c None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:27.739816 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86493401-95fb-444c-a905-7f31902f771c None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:27.740296 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86493401-95fb-444c-a905-7f31902f771c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:27.762019 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-86493401-95fb-444c-a905-7f31902f771c None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:25:27.763577 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-86493401-95fb-444c-a905-7f31902f771c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:25:27.845482 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 43/99] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:25:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 22:25:27.868756 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3b74aa-4fdb-4ee1-b026-c162b37507a8 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:27.917696 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:27.998278 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3b74aa-4fdb-4ee1-b026-c162b37507a8 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:28.044526 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5dd157-e021-43ff-bbdb-af062154b65d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:28.045797 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5dd157-e021-43ff-bbdb-af062154b65d None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:28.046090 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5dd157-e021-43ff-bbdb-af062154b65d None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:28.046373 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5dd157-e021-43ff-bbdb-af062154b65d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:28.071103 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5dd157-e021-43ff-bbdb-af062154b65d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 22:25:28.079377 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-8e5dd157-e021-43ff-bbdb-af062154b65d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 22:25:28.079377 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:25:28.079377 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:25:28.079377 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:25:28.079377 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:25:28.079377 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:25:28.079377 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:25:28.079377 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:25:28.079377 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:25:28.079377 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:25:28.079377 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:25:28.079377 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:25:28.079377 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 22:25:28.079377 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 22:25:28.079377 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 22:25:28.079377 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 22:25:28.080222 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 22:25:28.080222 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 22:25:28.080222 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 22:25:28.080222 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 22:25:28.080222 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:25:28.080222 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:25:28.080222 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:25:28.080222 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:25:28.080222 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:25:28.080222 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:25:28.080222 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:25:28.080222 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:25:28.080222 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:25:28.080222 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:25:28.080222 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:25:28.080222 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:25:28.080978 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:25:28.080978 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:25:28.080978 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:25:28.080978 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:25:28.080978 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 22:25:28.080978 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 22:25:28.080978 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 22:25:28.080978 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 22:25:28.080978 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 22:25:28.080978 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 22:25:28.080978 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 22:25:28.080978 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:25:28.080978 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 57/100] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Jul 20 22:25:27 2022] GET /identity/v3/roles/reader => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:25:28.086482 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86493401-95fb-444c-a905-7f31902f771c None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:28.121821 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c28509-484b-40bc-9201-c247a508b4ee None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:28.122114 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c28509-484b-40bc-9201-c247a508b4ee None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:28.122400 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c28509-484b-40bc-9201-c247a508b4ee None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:28.128051 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 58/101] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:25:28 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:25:28.133478 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f718bf3-b268-4aeb-9ed1-5fb5ff2b1807 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:28.133770 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f718bf3-b268-4aeb-9ed1-5fb5ff2b1807 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:28.134058 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f718bf3-b268-4aeb-9ed1-5fb5ff2b1807 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:28.172714 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-2f718bf3-b268-4aeb-9ed1-5fb5ff2b1807 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:25:28.173930 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-2f718bf3-b268-4aeb-9ed1-5fb5ff2b1807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:25:28.293857 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86493401-95fb-444c-a905-7f31902f771c None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:28.310077 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 59/102] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:25:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 22:25:28.311448 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e071d16b-0f41-4edb-94c3-3de19bba3611 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:28.311976 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e071d16b-0f41-4edb-94c3-3de19bba3611 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:28.312474 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e071d16b-0f41-4edb-94c3-3de19bba3611 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:28.314063 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 60/103] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:25:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:25:28.325090 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:28.326427 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfec2070-fabe-4d43-82d4-a19f5676c6dc None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:28.326939 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfec2070-fabe-4d43-82d4-a19f5676c6dc None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:28.327439 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfec2070-fabe-4d43-82d4-a19f5676c6dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:28.354984 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-cfec2070-fabe-4d43-82d4-a19f5676c6dc None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:25:28.356880 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-cfec2070-fabe-4d43-82d4-a19f5676c6dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:25:28.358236 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa51d346-8de1-4223-96f3-0faebabadcb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:28.359169 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa51d346-8de1-4223-96f3-0faebabadcb0 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:28.359660 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa51d346-8de1-4223-96f3-0faebabadcb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:28.360146 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa51d346-8de1-4223-96f3-0faebabadcb0 None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:28.361105 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa51d346-8de1-4223-96f3-0faebabadcb0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:25:28.361764 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa51d346-8de1-4223-96f3-0faebabadcb0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:28.363381 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa51d346-8de1-4223-96f3-0faebabadcb0 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:28.381465 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 44/104] 10.222.0.81 () {56 vars in 1101 bytes} [Wed Jul 20 22:25:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:25:28.382627 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa51d346-8de1-4223-96f3-0faebabadcb0 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:28.413455 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:28.470525 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 61/105] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:25:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 22:25:28.471833 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfec2070-fabe-4d43-82d4-a19f5676c6dc None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:28.512325 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa51d346-8de1-4223-96f3-0faebabadcb0 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:28.554019 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e12cb7-a224-4cfc-9fc6-b3eb0daad6c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:28.554766 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e12cb7-a224-4cfc-9fc6-b3eb0daad6c0 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:28.555071 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e12cb7-a224-4cfc-9fc6-b3eb0daad6c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:28.555355 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e12cb7-a224-4cfc-9fc6-b3eb0daad6c0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:28.565922 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e12cb7-a224-4cfc-9fc6-b3eb0daad6c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 22:25:28.574183 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-87e12cb7-a224-4cfc-9fc6-b3eb0daad6c0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 22:25:28.574183 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:25:28.574183 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:25:28.574183 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:25:28.574183 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:25:28.574183 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:25:28.574183 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:25:28.574183 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:25:28.574183 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:25:28.574183 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:25:28.574183 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:25:28.574183 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:25:28.574183 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 22:25:28.574183 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 22:25:28.574183 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 22:25:28.574183 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 22:25:28.575058 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 22:25:28.575058 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 22:25:28.575058 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 22:25:28.575058 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 22:25:28.575058 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:25:28.575058 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:25:28.575058 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:25:28.575058 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:25:28.575058 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:25:28.575058 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:25:28.575058 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:25:28.575058 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:25:28.575058 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:25:28.575058 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:25:28.575058 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:25:28.575058 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:25:28.575810 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:25:28.575810 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:25:28.575810 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:25:28.575810 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:25:28.575810 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 22:25:28.575810 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 22:25:28.575810 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 22:25:28.575810 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 22:25:28.575810 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 22:25:28.575810 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 22:25:28.575810 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 22:25:28.575810 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:25:28.575810 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 45/106] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Jul 20 22:25:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:25:28.576899 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff30d08c-2e25-4172-a22d-4add6ea40dcf None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:28.577000 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff30d08c-2e25-4172-a22d-4add6ea40dcf None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:28.577251 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff30d08c-2e25-4172-a22d-4add6ea40dcf None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:28.578764 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 46/107] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:25:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:25:28.583467 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff30d08c-2e25-4172-a22d-4add6ea40dcf None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:28.611563 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfec2070-fabe-4d43-82d4-a19f5676c6dc None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:28.630731 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:28.664696 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fa39d5-8ada-4486-b6eb-6824f3f4535a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:28.666415 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fa39d5-8ada-4486-b6eb-6824f3f4535a None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:28.666941 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fa39d5-8ada-4486-b6eb-6824f3f4535a None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:28.667844 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fa39d5-8ada-4486-b6eb-6824f3f4535a None admin] PATH_INFO: `/v3/users/1d8eed80f6204aa4b05b6f0d3edd0d8b` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:28.687391 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fa39d5-8ada-4486-b6eb-6824f3f4535a None admin] RBAC: Authorizing `identity:get_user(user_id=1d8eed80f6204aa4b05b6f0d3edd0d8b)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:28.689885 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fa39d5-8ada-4486-b6eb-6824f3f4535a None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:28.692185 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 62/108] 10.222.0.81 () {56 vars in 1177 bytes} [Wed Jul 20 22:25:28 2022] GET /identity/v3/users/1d8eed80f6204aa4b05b6f0d3edd0d8b => generated 289 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:25:28.693305 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fa39d5-8ada-4486-b6eb-6824f3f4535a None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:28.693611 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff30d08c-2e25-4172-a22d-4add6ea40dcf None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:28.719541 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:28.731056 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84e0bcf-f6f2-49c4-9ca1-04cec63c795c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:28.732117 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84e0bcf-f6f2-49c4-9ca1-04cec63c795c None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:28.732478 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84e0bcf-f6f2-49c4-9ca1-04cec63c795c None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:28.732832 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84e0bcf-f6f2-49c4-9ca1-04cec63c795c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:28.734439 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84e0bcf-f6f2-49c4-9ca1-04cec63c795c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:28.735980 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84e0bcf-f6f2-49c4-9ca1-04cec63c795c None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:28.742396 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 47/109] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Jul 20 22:25:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:25:28.742396 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c15a428-1dfd-4973-aca9-ac9e4a9242e9 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:28.742396 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c15a428-1dfd-4973-aca9-ac9e4a9242e9 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:28.742396 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c15a428-1dfd-4973-aca9-ac9e4a9242e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:28.764954 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-6c15a428-1dfd-4973-aca9-ac9e4a9242e9 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:25:28.766161 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-6c15a428-1dfd-4973-aca9-ac9e4a9242e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:25:28.818320 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fa39d5-8ada-4486-b6eb-6824f3f4535a None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:28.887033 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 48/110] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:25:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 22:25:28.887219 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e056855c-b942-4c4f-aa92-49a635f8182e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:28.887219 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e056855c-b942-4c4f-aa92-49a635f8182e None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:28.887405 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c15a428-1dfd-4973-aca9-ac9e4a9242e9 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:28.887498 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e056855c-b942-4c4f-aa92-49a635f8182e None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:28.887498 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e056855c-b942-4c4f-aa92-49a635f8182e None admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:28.887498 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e056855c-b942-4c4f-aa92-49a635f8182e None admin] RBAC: Adding query filter params (name=member) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:25:28.887498 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e056855c-b942-4c4f-aa92-49a635f8182e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:28.887498 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e056855c-b942-4c4f-aa92-49a635f8182e None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:28.896792 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 63/111] 10.222.0.81 () {56 vars in 1101 bytes} [Wed Jul 20 22:25:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:25:28.897648 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e056855c-b942-4c4f-aa92-49a635f8182e None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:28.931452 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:28.987889 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:28.990273 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c15a428-1dfd-4973-aca9-ac9e4a9242e9 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:29.007739 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e056855c-b942-4c4f-aa92-49a635f8182e None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:29.026719 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1437b050-4475-426e-ba64-da19ad39c346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:29.027458 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1437b050-4475-426e-ba64-da19ad39c346 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:29.027746 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1437b050-4475-426e-ba64-da19ad39c346 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:29.028383 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1437b050-4475-426e-ba64-da19ad39c346 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:29.043456 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a1d18b-467c-4815-874a-b8f74f74fe1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:29.044155 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a1d18b-467c-4815-874a-b8f74f74fe1d None admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:29.044447 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a1d18b-467c-4815-874a-b8f74f74fe1d None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:29.044744 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a1d18b-467c-4815-874a-b8f74f74fe1d None admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:29.045728 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a1d18b-467c-4815-874a-b8f74f74fe1d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:29.047194 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a1d18b-467c-4815-874a-b8f74f74fe1d None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:29.098478 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1437b050-4475-426e-ba64-da19ad39c346 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:29.099930 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1437b050-4475-426e-ba64-da19ad39c346 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:29.139080 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:29.192749 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 64/112] 10.222.0.81 () {60 vars in 1141 bytes} [Wed Jul 20 22:25:28 2022] POST /identity/v3/projects => generated 302 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:25:29.204753 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38843ae-464e-415e-a579-e45b24b68265 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:29.205122 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38843ae-464e-415e-a579-e45b24b68265 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:29.205420 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38843ae-464e-415e-a579-e45b24b68265 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:29.207114 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 65/113] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:25:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:25:29.207837 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38843ae-464e-415e-a579-e45b24b68265 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:29.219606 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 49/114] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Jul 20 22:25:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 22:25:29.219606 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1437b050-4475-426e-ba64-da19ad39c346 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:29.356726 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38843ae-464e-415e-a579-e45b24b68265 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:29.361732 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1437b050-4475-426e-ba64-da19ad39c346 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:29.390888 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:29.396067 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e68489b-87fa-4848-ac78-0c9faf9ddf7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:29.398541 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e68489b-87fa-4848-ac78-0c9faf9ddf7a None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:29.398541 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e68489b-87fa-4848-ac78-0c9faf9ddf7a None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:29.398541 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e68489b-87fa-4848-ac78-0c9faf9ddf7a None admin] PATH_INFO: `/v3/users/74bf4b54c7664f9fac20de5b75d465f8` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:29.398776 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e68489b-87fa-4848-ac78-0c9faf9ddf7a None admin] RBAC: Authorizing `identity:get_user(user_id=74bf4b54c7664f9fac20de5b75d465f8)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:29.401785 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e68489b-87fa-4848-ac78-0c9faf9ddf7a None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:29.409766 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 66/115] 10.222.0.81 () {56 vars in 1177 bytes} [Wed Jul 20 22:25:29 2022] GET /identity/v3/users/74bf4b54c7664f9fac20de5b75d465f8 => generated 275 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:25:29.409766 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90e8b2e-ce91-420e-ac9a-b035e981c58a None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:29.409766 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90e8b2e-ce91-420e-ac9a-b035e981c58a None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:29.409766 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90e8b2e-ce91-420e-ac9a-b035e981c58a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:29.410058 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deedc0b0-aa26-4aff-ba35-48c8dc440f1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:29.410058 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deedc0b0-aa26-4aff-ba35-48c8dc440f1a None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:29.410975 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deedc0b0-aa26-4aff-ba35-48c8dc440f1a None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:29.411313 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deedc0b0-aa26-4aff-ba35-48c8dc440f1a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:29.423682 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-a90e8b2e-ce91-420e-ac9a-b035e981c58a None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:25:29.424449 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:29.424906 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-a90e8b2e-ce91-420e-ac9a-b035e981c58a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:25:29.425757 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deedc0b0-aa26-4aff-ba35-48c8dc440f1a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 22:25:29.435594 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-deedc0b0-aa26-4aff-ba35-48c8dc440f1a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 22:25:29.435594 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:25:29.435594 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:25:29.435594 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:25:29.435594 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:25:29.435594 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:25:29.435594 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:25:29.435594 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:25:29.435594 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:25:29.435594 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:25:29.435594 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:25:29.435594 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:25:29.435594 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 22:25:29.435594 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 22:25:29.435594 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 22:25:29.435594 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 22:25:29.436649 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 22:25:29.436649 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 22:25:29.436649 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 22:25:29.436649 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 22:25:29.436649 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:25:29.436649 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:25:29.436649 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:25:29.436649 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:25:29.436649 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:25:29.436649 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:25:29.436649 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:25:29.436649 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:25:29.436649 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:25:29.436649 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:25:29.436649 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:25:29.436649 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:25:29.439178 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:25:29.439178 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:25:29.439178 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:25:29.439178 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:25:29.439178 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 22:25:29.439178 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 22:25:29.439178 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 22:25:29.439178 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 22:25:29.439178 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 22:25:29.439178 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 22:25:29.439178 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 22:25:29.439178 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:25:29.439178 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 50/116] 10.222.0.81 () {56 vars in 1096 bytes} [Wed Jul 20 22:25:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:25:29.439178 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deedc0b0-aa26-4aff-ba35-48c8dc440f1a None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:29.507068 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 67/117] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:25:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 22:25:29.507068 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90e8b2e-ce91-420e-ac9a-b035e981c58a None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:29.547662 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deedc0b0-aa26-4aff-ba35-48c8dc440f1a None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:29.583163 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba9e2e8-9379-469c-9b20-3f5b869d22db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:29.584168 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba9e2e8-9379-469c-9b20-3f5b869d22db None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:29.584571 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba9e2e8-9379-469c-9b20-3f5b869d22db None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:29.584853 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba9e2e8-9379-469c-9b20-3f5b869d22db None admin] PATH_INFO: `/v3/projects/da54f6860e1f43c29cf90271280bdcae` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:29.586066 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba9e2e8-9379-469c-9b20-3f5b869d22db None admin] RBAC: Authorizing `identity:get_project(project_id=da54f6860e1f43c29cf90271280bdcae)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:29.587565 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba9e2e8-9379-469c-9b20-3f5b869d22db None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:29.589727 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 51/118] 10.222.0.81 () {56 vars in 1186 bytes} [Wed Jul 20 22:25:29 2022] GET /identity/v3/projects/da54f6860e1f43c29cf90271280bdcae => generated 298 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:25:29.591034 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b649bfc-3c18-4da4-b2c4-12f994b7b864 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:29.591329 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b649bfc-3c18-4da4-b2c4-12f994b7b864 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:29.591611 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b649bfc-3c18-4da4-b2c4-12f994b7b864 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:29.592971 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 52/119] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:25:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:25:29.593532 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b649bfc-3c18-4da4-b2c4-12f994b7b864 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:29.623144 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90e8b2e-ce91-420e-ac9a-b035e981c58a None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:29.659108 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:29.665682 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:29.666005 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a12659-c2a8-4e92-8882-5c7d85b1fd34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:29.667846 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a12659-c2a8-4e92-8882-5c7d85b1fd34 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:29.677835 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a12659-c2a8-4e92-8882-5c7d85b1fd34 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:29.677835 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a12659-c2a8-4e92-8882-5c7d85b1fd34 None admin] PATH_INFO: `/v3/projects/da54f6860e1f43c29cf90271280bdcae` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:29.677835 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a12659-c2a8-4e92-8882-5c7d85b1fd34 None admin] RBAC: Authorizing `identity:get_project(project_id=da54f6860e1f43c29cf90271280bdcae)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:29.677835 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a12659-c2a8-4e92-8882-5c7d85b1fd34 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:29.677835 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 68/120] 10.222.0.81 () {56 vars in 1186 bytes} [Wed Jul 20 22:25:29 2022] GET /identity/v3/projects/da54f6860e1f43c29cf90271280bdcae => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:25:29.677835 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a12659-c2a8-4e92-8882-5c7d85b1fd34 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:29.701818 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b649bfc-3c18-4da4-b2c4-12f994b7b864 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:29.736237 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2b77aa-1611-4263-bdb2-1e7a3f9623eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:29.736921 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2b77aa-1611-4263-bdb2-1e7a3f9623eb None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:29.737216 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2b77aa-1611-4263-bdb2-1e7a3f9623eb None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:29.737497 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2b77aa-1611-4263-bdb2-1e7a3f9623eb None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:29.738243 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2b77aa-1611-4263-bdb2-1e7a3f9623eb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:25:29.738558 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2b77aa-1611-4263-bdb2-1e7a3f9623eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:29.739986 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2b77aa-1611-4263-bdb2-1e7a3f9623eb None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:29.749037 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 53/121] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Jul 20 22:25:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:25:29.749846 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2b77aa-1611-4263-bdb2-1e7a3f9623eb None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:29.788288 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a12659-c2a8-4e92-8882-5c7d85b1fd34 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:29.823307 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e567727d-75be-4799-ad26-485eda23b849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:29.824795 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e567727d-75be-4799-ad26-485eda23b849 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:29.825251 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e567727d-75be-4799-ad26-485eda23b849 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:29.825688 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e567727d-75be-4799-ad26-485eda23b849 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:29.837039 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e567727d-75be-4799-ad26-485eda23b849 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 22:25:29.846227 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-e567727d-75be-4799-ad26-485eda23b849 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 22:25:29.846227 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:25:29.846227 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:25:29.846227 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:25:29.846227 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:25:29.846227 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:25:29.846227 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:25:29.846227 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:25:29.846227 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:25:29.846227 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:25:29.846227 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:25:29.846227 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:25:29.846227 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 22:25:29.846227 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 22:25:29.846227 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 22:25:29.846227 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 22:25:29.847262 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 22:25:29.847262 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 22:25:29.847262 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 22:25:29.847262 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 22:25:29.847262 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:25:29.847262 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:25:29.847262 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:25:29.847262 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:25:29.847262 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:25:29.847262 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:25:29.847262 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:25:29.847262 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:25:29.847262 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:25:29.847262 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:25:29.847262 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:25:29.847262 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:25:29.848208 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:25:29.848208 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:25:29.848208 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:25:29.848208 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:25:29.848208 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 22:25:29.848208 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 22:25:29.848208 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 22:25:29.848208 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 22:25:29.848208 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 22:25:29.848208 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 22:25:29.848208 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 22:25:29.848208 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:25:29.850132 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 69/122] 10.222.0.81 () {56 vars in 1114 bytes} [Wed Jul 20 22:25:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:25:29.851487 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c13700-7a75-4f63-82ae-f0beb2cdea3c None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:29.851772 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c13700-7a75-4f63-82ae-f0beb2cdea3c None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:29.852053 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c13700-7a75-4f63-82ae-f0beb2cdea3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:29.859509 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2b77aa-1611-4263-bdb2-1e7a3f9623eb None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:29.869388 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-12c13700-7a75-4f63-82ae-f0beb2cdea3c None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:25:29.870867 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-12c13700-7a75-4f63-82ae-f0beb2cdea3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:25:29.888594 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:29.895124 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59db6b52-7707-4ebe-8834-97f8f270988e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:29.895806 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59db6b52-7707-4ebe-8834-97f8f270988e None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:29.896081 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59db6b52-7707-4ebe-8834-97f8f270988e None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:29.896373 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59db6b52-7707-4ebe-8834-97f8f270988e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:29.897186 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59db6b52-7707-4ebe-8834-97f8f270988e None admin] RBAC: Adding query filter params (role.id=5b2ef0feadf94c059dffbd7d8bc03aa4, scope.project.id=da54f6860e1f43c29cf90271280bdcae, user.id=1d8eed80f6204aa4b05b6f0d3edd0d8b) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:25:29.897509 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59db6b52-7707-4ebe-8834-97f8f270988e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:29.898982 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59db6b52-7707-4ebe-8834-97f8f270988e None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:29.915040 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 54/123] 10.222.0.81 () {56 vars in 1374 bytes} [Wed Jul 20 22:25:29 2022] GET /identity/v3/role_assignments?user.id=1d8eed80f6204aa4b05b6f0d3edd0d8b&scope.project.id=da54f6860e1f43c29cf90271280bdcae&role.id=5b2ef0feadf94c059dffbd7d8bc03aa4 => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:25:29.915040 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59db6b52-7707-4ebe-8834-97f8f270988e None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:29.941347 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:29.964515 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 70/124] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:25:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 22:25:29.965781 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c13700-7a75-4f63-82ae-f0beb2cdea3c None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:30.068754 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59db6b52-7707-4ebe-8834-97f8f270988e None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:30.091754 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c13700-7a75-4f63-82ae-f0beb2cdea3c None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:30.109110 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a22e6b1-df3d-459c-ac70-7ea70cbc0683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:30.110528 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a22e6b1-df3d-459c-ac70-7ea70cbc0683 None admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:30.111041 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a22e6b1-df3d-459c-ac70-7ea70cbc0683 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:30.111520 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a22e6b1-df3d-459c-ac70-7ea70cbc0683 None admin] PATH_INFO: `/v3/projects/da54f6860e1f43c29cf90271280bdcae/users/74bf4b54c7664f9fac20de5b75d465f8/roles/559942c4de1744169f8b5e0404ecde6b` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:30.113859 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a22e6b1-df3d-459c-ac70-7ea70cbc0683 None admin] RBAC: Authorizing `identity:create_grant(project_id=da54f6860e1f43c29cf90271280bdcae, user_id=74bf4b54c7664f9fac20de5b75d465f8, role_id=559942c4de1744169f8b5e0404ecde6b)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:30.115746 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a22e6b1-df3d-459c-ac70-7ea70cbc0683 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:30.129701 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0e72b3-3b9d-4728-8afb-1b13f043fc6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:30.130604 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0e72b3-3b9d-4728-8afb-1b13f043fc6c None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:30.130896 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0e72b3-3b9d-4728-8afb-1b13f043fc6c None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:30.131208 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0e72b3-3b9d-4728-8afb-1b13f043fc6c None admin] PATH_INFO: `/v3/users/98f5d4c4be4845478aa0d31a1403d775` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:30.132406 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0e72b3-3b9d-4728-8afb-1b13f043fc6c None admin] RBAC: Authorizing `identity:get_user(user_id=98f5d4c4be4845478aa0d31a1403d775)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:30.133860 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0e72b3-3b9d-4728-8afb-1b13f043fc6c None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:30.139182 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 71/125] 10.222.0.81 () {56 vars in 1177 bytes} [Wed Jul 20 22:25:29 2022] GET /identity/v3/users/98f5d4c4be4845478aa0d31a1403d775 => generated 247 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:25:30.139182 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0e72b3-3b9d-4728-8afb-1b13f043fc6c None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:30.197467 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 55/126] 10.222.0.81 () {58 vars in 1439 bytes} [Wed Jul 20 22:25:29 2022] PUT /identity/v3/projects/da54f6860e1f43c29cf90271280bdcae/users/74bf4b54c7664f9fac20de5b75d465f8/roles/559942c4de1744169f8b5e0404ecde6b => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:25:30.207058 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a22e6b1-df3d-459c-ac70-7ea70cbc0683 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:30.215177 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:30.275436 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:30.321175 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a22e6b1-df3d-459c-ac70-7ea70cbc0683 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:30.326436 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0e72b3-3b9d-4728-8afb-1b13f043fc6c None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:30.366059 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd0fabe-0283-4da4-a130-6e970b0ff79c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:30.368266 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd0fabe-0283-4da4-a130-6e970b0ff79c None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:30.368563 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd0fabe-0283-4da4-a130-6e970b0ff79c None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:30.368846 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd0fabe-0283-4da4-a130-6e970b0ff79c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:30.392516 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd0fabe-0283-4da4-a130-6e970b0ff79c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 22:25:30.397743 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933568b1-88da-411b-898c-20b28a301719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:30.398672 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933568b1-88da-411b-898c-20b28a301719 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:30.399200 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933568b1-88da-411b-898c-20b28a301719 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:30.399685 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933568b1-88da-411b-898c-20b28a301719 None admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:30.400664 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933568b1-88da-411b-898c-20b28a301719 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:25:30.401736 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933568b1-88da-411b-898c-20b28a301719 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:30.402018 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-acd0fabe-0283-4da4-a130-6e970b0ff79c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 22:25:30.402018 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:25:30.402018 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:25:30.402018 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:25:30.402018 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:25:30.402018 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:25:30.402018 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:25:30.402018 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:25:30.402018 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:25:30.402018 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:25:30.402018 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:25:30.402018 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:25:30.402018 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 22:25:30.402018 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 22:25:30.402018 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 22:25:30.402018 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 22:25:30.402878 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 22:25:30.402878 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 22:25:30.402878 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 22:25:30.402878 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 22:25:30.402878 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:25:30.402878 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:25:30.402878 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:25:30.402878 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:25:30.402878 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:25:30.402878 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:25:30.402878 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:25:30.402878 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:25:30.402878 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:25:30.402878 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:25:30.402878 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:25:30.402878 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:25:30.403675 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:25:30.403675 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:25:30.403675 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:25:30.403675 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:25:30.403675 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 22:25:30.403675 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 22:25:30.403675 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 22:25:30.403675 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 22:25:30.403675 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 22:25:30.403675 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 22:25:30.403675 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 22:25:30.403675 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:25:30.404266 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 56/127] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Jul 20 22:25:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:25:30.405080 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd0fabe-0283-4da4-a130-6e970b0ff79c None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:30.406755 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933568b1-88da-411b-898c-20b28a301719 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:30.418520 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 72/128] 10.222.0.81 () {56 vars in 1111 bytes} [Wed Jul 20 22:25:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:25:30.419796 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933568b1-88da-411b-898c-20b28a301719 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:30.561026 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933568b1-88da-411b-898c-20b28a301719 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:30.581486 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:30.583787 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd0fabe-0283-4da4-a130-6e970b0ff79c None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:30.602907 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1836b898-fc7d-49fe-b12d-15ed438faaf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:30.603973 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1836b898-fc7d-49fe-b12d-15ed438faaf7 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:30.605020 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1836b898-fc7d-49fe-b12d-15ed438faaf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:30.605506 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1836b898-fc7d-49fe-b12d-15ed438faaf7 None admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:30.606927 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1836b898-fc7d-49fe-b12d-15ed438faaf7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:25:30.607472 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1836b898-fc7d-49fe-b12d-15ed438faaf7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:30.609453 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1836b898-fc7d-49fe-b12d-15ed438faaf7 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:30.612472 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:30.622492 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 73/129] 10.222.0.81 () {56 vars in 1101 bytes} [Wed Jul 20 22:25:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:25:30.623616 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1836b898-fc7d-49fe-b12d-15ed438faaf7 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:30.668011 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab57b0ed-8f8e-43c0-ad23-72c90b5ee3a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:30.668882 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab57b0ed-8f8e-43c0-ad23-72c90b5ee3a5 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:30.669175 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab57b0ed-8f8e-43c0-ad23-72c90b5ee3a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:30.669459 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab57b0ed-8f8e-43c0-ad23-72c90b5ee3a5 None admin] PATH_INFO: `/v3/projects/da54f6860e1f43c29cf90271280bdcae` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:30.672741 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab57b0ed-8f8e-43c0-ad23-72c90b5ee3a5 None admin] RBAC: Authorizing `identity:get_project(project_id=da54f6860e1f43c29cf90271280bdcae)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:30.674281 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab57b0ed-8f8e-43c0-ad23-72c90b5ee3a5 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:30.676037 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 57/130] 10.222.0.81 () {56 vars in 1186 bytes} [Wed Jul 20 22:25:30 2022] GET /identity/v3/projects/da54f6860e1f43c29cf90271280bdcae => generated 298 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:25:30.676862 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab57b0ed-8f8e-43c0-ad23-72c90b5ee3a5 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:30.744639 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1836b898-fc7d-49fe-b12d-15ed438faaf7 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:30.784978 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979c5569-d51f-4130-90ef-a91579c4c337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:30.786274 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979c5569-d51f-4130-90ef-a91579c4c337 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:30.787364 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979c5569-d51f-4130-90ef-a91579c4c337 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:30.787856 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979c5569-d51f-4130-90ef-a91579c4c337 None admin] PATH_INFO: `/v3/users/74bf4b54c7664f9fac20de5b75d465f8` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:30.789332 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979c5569-d51f-4130-90ef-a91579c4c337 None admin] RBAC: Authorizing `identity:get_user(user_id=74bf4b54c7664f9fac20de5b75d465f8)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:30.790984 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979c5569-d51f-4130-90ef-a91579c4c337 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:30.794779 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 74/131] 10.222.0.81 () {56 vars in 1177 bytes} [Wed Jul 20 22:25:30 2022] GET /identity/v3/users/74bf4b54c7664f9fac20de5b75d465f8 => generated 275 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:25:30.796405 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979c5569-d51f-4130-90ef-a91579c4c337 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:30.858674 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:30.905499 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab57b0ed-8f8e-43c0-ad23-72c90b5ee3a5 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:30.926588 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979c5569-d51f-4130-90ef-a91579c4c337 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:30.942626 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad90753b-292b-425f-ad74-2bde6f336659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:30.943754 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad90753b-292b-425f-ad74-2bde6f336659 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:30.943963 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad90753b-292b-425f-ad74-2bde6f336659 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:30.944247 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad90753b-292b-425f-ad74-2bde6f336659 None admin] PATH_INFO: `/v3/users/74bf4b54c7664f9fac20de5b75d465f8` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:30.945494 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad90753b-292b-425f-ad74-2bde6f336659 None admin] RBAC: Authorizing `identity:get_user(user_id=74bf4b54c7664f9fac20de5b75d465f8)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:30.947056 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad90753b-292b-425f-ad74-2bde6f336659 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:30.949000 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 58/132] 10.222.0.81 () {56 vars in 1177 bytes} [Wed Jul 20 22:25:30 2022] GET /identity/v3/users/74bf4b54c7664f9fac20de5b75d465f8 => generated 275 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:25:30.949779 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad90753b-292b-425f-ad74-2bde6f336659 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:30.967056 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d95641-ae81-47ef-a029-7518fe4aab84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:30.968306 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d95641-ae81-47ef-a029-7518fe4aab84 None admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:30.968797 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d95641-ae81-47ef-a029-7518fe4aab84 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:30.969392 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d95641-ae81-47ef-a029-7518fe4aab84 None admin] PATH_INFO: `/v3/projects/da54f6860e1f43c29cf90271280bdcae/users/98f5d4c4be4845478aa0d31a1403d775/roles/d89b6c92d35a401a8517f1d47f6be843` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:30.971721 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d95641-ae81-47ef-a029-7518fe4aab84 None admin] RBAC: Authorizing `identity:create_grant(project_id=da54f6860e1f43c29cf90271280bdcae, user_id=98f5d4c4be4845478aa0d31a1403d775, role_id=d89b6c92d35a401a8517f1d47f6be843)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:30.973495 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d95641-ae81-47ef-a029-7518fe4aab84 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:31.007361 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:31.064178 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad90753b-292b-425f-ad74-2bde6f336659 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:31.101091 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345f69cc-2bc9-4893-93ea-194b6734b75f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:31.101997 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345f69cc-2bc9-4893-93ea-194b6734b75f None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:31.102288 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345f69cc-2bc9-4893-93ea-194b6734b75f None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:31.102568 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345f69cc-2bc9-4893-93ea-194b6734b75f None admin] PATH_INFO: `/v3/projects/da54f6860e1f43c29cf90271280bdcae` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:31.104539 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345f69cc-2bc9-4893-93ea-194b6734b75f None admin] RBAC: Authorizing `identity:get_project(project_id=da54f6860e1f43c29cf90271280bdcae)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:31.106065 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345f69cc-2bc9-4893-93ea-194b6734b75f None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:31.107969 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 59/133] 10.222.0.81 () {56 vars in 1186 bytes} [Wed Jul 20 22:25:30 2022] GET /identity/v3/projects/da54f6860e1f43c29cf90271280bdcae => generated 298 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:25:31.108797 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345f69cc-2bc9-4893-93ea-194b6734b75f None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:31.129242 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 75/134] 10.222.0.81 () {58 vars in 1439 bytes} [Wed Jul 20 22:25:30 2022] PUT /identity/v3/projects/da54f6860e1f43c29cf90271280bdcae/users/98f5d4c4be4845478aa0d31a1403d775/roles/d89b6c92d35a401a8517f1d47f6be843 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:25:31.130549 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d95641-ae81-47ef-a029-7518fe4aab84 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:31.335871 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345f69cc-2bc9-4893-93ea-194b6734b75f None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:31.361241 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d95641-ae81-47ef-a029-7518fe4aab84 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:31.371366 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:31.377610 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:31.398405 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3987d60d-c6fe-4c23-8a53-16d44df069cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:31.399332 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3987d60d-c6fe-4c23-8a53-16d44df069cd None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:31.399622 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3987d60d-c6fe-4c23-8a53-16d44df069cd None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:31.399905 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3987d60d-c6fe-4c23-8a53-16d44df069cd None admin] PATH_INFO: `/v3/projects/311b0d7cef0c483396ad292837f6d314` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:31.402302 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3987d60d-c6fe-4c23-8a53-16d44df069cd None admin] RBAC: Authorizing `identity:get_project(project_id=311b0d7cef0c483396ad292837f6d314)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:31.403782 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3987d60d-c6fe-4c23-8a53-16d44df069cd None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:31.405666 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 60/135] 10.222.0.81 () {56 vars in 1186 bytes} [Wed Jul 20 22:25:31 2022] GET /identity/v3/projects/311b0d7cef0c483396ad292837f6d314 => generated 312 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:25:31.411968 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3987d60d-c6fe-4c23-8a53-16d44df069cd None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:31.426097 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec5548e-30d5-45ea-94f5-ee6379cfbbc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:31.427340 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec5548e-30d5-45ea-94f5-ee6379cfbbc0 None admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:31.427857 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec5548e-30d5-45ea-94f5-ee6379cfbbc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:31.428298 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec5548e-30d5-45ea-94f5-ee6379cfbbc0 None admin] PATH_INFO: `/v3/projects/da54f6860e1f43c29cf90271280bdcae/users/74bf4b54c7664f9fac20de5b75d465f8/roles/6d5a3e6a22fa4f4fa47afcfce0246a1f` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:31.433880 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec5548e-30d5-45ea-94f5-ee6379cfbbc0 None admin] RBAC: Authorizing `identity:create_grant(project_id=da54f6860e1f43c29cf90271280bdcae, user_id=74bf4b54c7664f9fac20de5b75d465f8, role_id=6d5a3e6a22fa4f4fa47afcfce0246a1f)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:31.435676 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec5548e-30d5-45ea-94f5-ee6379cfbbc0 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:31.513743 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 76/136] 10.222.0.81 () {58 vars in 1439 bytes} [Wed Jul 20 22:25:31 2022] PUT /identity/v3/projects/da54f6860e1f43c29cf90271280bdcae/users/74bf4b54c7664f9fac20de5b75d465f8/roles/6d5a3e6a22fa4f4fa47afcfce0246a1f => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:25:31.540248 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3987d60d-c6fe-4c23-8a53-16d44df069cd None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:31.573657 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab95cd82-4b0a-40e4-ad88-e28d2a65664d None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:31.574229 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab95cd82-4b0a-40e4-ad88-e28d2a65664d None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:31.574693 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab95cd82-4b0a-40e4-ad88-e28d2a65664d None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:31.576822 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 77/137] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:25:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 22:25:31.582232 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec828229-c74b-4922-a2f7-6a0f8c42a0d4 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:31.582757 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec828229-c74b-4922-a2f7-6a0f8c42a0d4 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:31.583255 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec828229-c74b-4922-a2f7-6a0f8c42a0d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:31.590897 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88c5585-064c-49f1-b3d0-1bafba320218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:31.592062 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88c5585-064c-49f1-b3d0-1bafba320218 None admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:31.592250 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88c5585-064c-49f1-b3d0-1bafba320218 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:31.592533 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88c5585-064c-49f1-b3d0-1bafba320218 None admin] PATH_INFO: `/v3/projects/311b0d7cef0c483396ad292837f6d314/users/74bf4b54c7664f9fac20de5b75d465f8/roles/559942c4de1744169f8b5e0404ecde6b` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:31.603362 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88c5585-064c-49f1-b3d0-1bafba320218 None admin] RBAC: Authorizing `identity:create_grant(project_id=311b0d7cef0c483396ad292837f6d314, user_id=74bf4b54c7664f9fac20de5b75d465f8, role_id=559942c4de1744169f8b5e0404ecde6b)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:31.603362 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88c5585-064c-49f1-b3d0-1bafba320218 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:31.616295 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-ec828229-c74b-4922-a2f7-6a0f8c42a0d4 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:25:31.617948 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-ec828229-c74b-4922-a2f7-6a0f8c42a0d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:25:31.682819 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 61/138] 10.222.0.81 () {58 vars in 1439 bytes} [Wed Jul 20 22:25:31 2022] PUT /identity/v3/projects/311b0d7cef0c483396ad292837f6d314/users/74bf4b54c7664f9fac20de5b75d465f8/roles/559942c4de1744169f8b5e0404ecde6b => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:25:31.720201 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 78/139] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:25:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 22:25:31.746326 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88c5585-064c-49f1-b3d0-1bafba320218 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:31.816633 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:31.882369 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88c5585-064c-49f1-b3d0-1bafba320218 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:31.919544 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b501ac-1b8f-4d65-a505-7a539a7176c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:31.920616 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b501ac-1b8f-4d65-a505-7a539a7176c2 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:31.920906 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b501ac-1b8f-4d65-a505-7a539a7176c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:31.921267 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b501ac-1b8f-4d65-a505-7a539a7176c2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:31.922473 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b501ac-1b8f-4d65-a505-7a539a7176c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:31.923984 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b501ac-1b8f-4d65-a505-7a539a7176c2 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:31.925901 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 62/140] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Jul 20 22:25:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:25:31.932003 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec828229-c74b-4922-a2f7-6a0f8c42a0d4 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:32.064475 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:32.076008 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec828229-c74b-4922-a2f7-6a0f8c42a0d4 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:32.114452 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b253f7-66a7-4b55-b890-e90236fa3c63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:32.115218 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b253f7-66a7-4b55-b890-e90236fa3c63 None admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:32.115513 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b253f7-66a7-4b55-b890-e90236fa3c63 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:32.115799 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b253f7-66a7-4b55-b890-e90236fa3c63 None admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:32.116594 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b253f7-66a7-4b55-b890-e90236fa3c63 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:32.118069 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b253f7-66a7-4b55-b890-e90236fa3c63 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:32.279638 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 79/141] 10.222.0.81 () {60 vars in 1132 bytes} [Wed Jul 20 22:25:31 2022] POST /identity/v3/users => generated 283 bytes in 348 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:25:32.460358 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bf42e1-09fd-48d0-a548-79c9ab3eb13e None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:32.460642 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bf42e1-09fd-48d0-a548-79c9ab3eb13e None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:32.460896 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bf42e1-09fd-48d0-a548-79c9ab3eb13e None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:32.462399 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 63/142] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:25:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:25:32.467477 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e308822-8db2-4a19-884f-f404c6bf92d3 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:32.467778 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e308822-8db2-4a19-884f-f404c6bf92d3 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:32.468065 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e308822-8db2-4a19-884f-f404c6bf92d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:32.491227 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-0e308822-8db2-4a19-884f-f404c6bf92d3 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:25:32.492430 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-0e308822-8db2-4a19-884f-f404c6bf92d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:25:32.584922 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 80/143] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:25:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 22:25:32.628110 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bf42e1-09fd-48d0-a548-79c9ab3eb13e None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:32.750991 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bf42e1-09fd-48d0-a548-79c9ab3eb13e None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:32.770365 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:32.792459 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd17bf19-0b5c-4878-98e1-b36ede6be83d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:32.793289 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd17bf19-0b5c-4878-98e1-b36ede6be83d None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:32.793579 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd17bf19-0b5c-4878-98e1-b36ede6be83d None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:32.793858 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd17bf19-0b5c-4878-98e1-b36ede6be83d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:32.807506 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd17bf19-0b5c-4878-98e1-b36ede6be83d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 22:25:32.816082 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-dd17bf19-0b5c-4878-98e1-b36ede6be83d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 22:25:32.816082 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:25:32.816082 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:25:32.816082 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:25:32.816082 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:25:32.816082 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:25:32.816082 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:25:32.816082 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:25:32.816082 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:25:32.816082 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:25:32.816082 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:25:32.816082 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:25:32.816082 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 22:25:32.816082 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 22:25:32.816082 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 22:25:32.816082 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 22:25:32.816933 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 22:25:32.816933 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 22:25:32.816933 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 22:25:32.816933 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 22:25:32.816933 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:25:32.816933 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:25:32.816933 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:25:32.816933 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:25:32.816933 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:25:32.816933 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:25:32.816933 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:25:32.816933 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:25:32.816933 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:25:32.816933 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:25:32.816933 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:25:32.816933 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:25:32.818111 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:25:32.818111 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:25:32.818111 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:25:32.818111 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:25:32.818111 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 22:25:32.818111 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 22:25:32.818111 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 22:25:32.818111 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 22:25:32.818111 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 22:25:32.818111 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 22:25:32.818111 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 22:25:32.818111 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:25:32.818111 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 64/144] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Jul 20 22:25:32 2022] GET /identity/v3/roles/reader => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:25:32.823681 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e308822-8db2-4a19-884f-f404c6bf92d3 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:32.921042 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:32.937204 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e308822-8db2-4a19-884f-f404c6bf92d3 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:32.983883 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbdec7e-3313-462e-ac7b-e041519f457d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:32.984570 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbdec7e-3313-462e-ac7b-e041519f457d None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:32.984861 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbdec7e-3313-462e-ac7b-e041519f457d None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:32.985143 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbdec7e-3313-462e-ac7b-e041519f457d None admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:32.985907 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbdec7e-3313-462e-ac7b-e041519f457d None admin] RBAC: Adding query filter params (name=reader) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:25:32.986217 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbdec7e-3313-462e-ac7b-e041519f457d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:32.987929 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbdec7e-3313-462e-ac7b-e041519f457d None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:33.000070 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 81/145] 10.222.0.81 () {56 vars in 1101 bytes} [Wed Jul 20 22:25:32 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:25:33.005621 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd17bf19-0b5c-4878-98e1-b36ede6be83d None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:33.124605 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd17bf19-0b5c-4878-98e1-b36ede6be83d None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:33.167383 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342f767f-6069-4020-91f2-7e59661e8290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:33.168675 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342f767f-6069-4020-91f2-7e59661e8290 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:33.169247 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342f767f-6069-4020-91f2-7e59661e8290 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:33.169990 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342f767f-6069-4020-91f2-7e59661e8290 None admin] PATH_INFO: `/v3/users/1d8eed80f6204aa4b05b6f0d3edd0d8b` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:33.171588 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342f767f-6069-4020-91f2-7e59661e8290 None admin] RBAC: Authorizing `identity:get_user(user_id=1d8eed80f6204aa4b05b6f0d3edd0d8b)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:33.173289 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342f767f-6069-4020-91f2-7e59661e8290 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:33.176204 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 65/146] 10.222.0.81 () {56 vars in 1177 bytes} [Wed Jul 20 22:25:33 2022] GET /identity/v3/users/1d8eed80f6204aa4b05b6f0d3edd0d8b => generated 289 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:25:33.186363 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbdec7e-3313-462e-ac7b-e041519f457d None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:33.319472 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbdec7e-3313-462e-ac7b-e041519f457d None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:33.355803 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd8c99e-c394-45af-91cd-75a066de783b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:33.356684 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd8c99e-c394-45af-91cd-75a066de783b None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:33.356970 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd8c99e-c394-45af-91cd-75a066de783b None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:33.357248 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd8c99e-c394-45af-91cd-75a066de783b None admin] PATH_INFO: `/v3/projects/da54f6860e1f43c29cf90271280bdcae` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:33.358456 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd8c99e-c394-45af-91cd-75a066de783b None admin] RBAC: Authorizing `identity:get_project(project_id=da54f6860e1f43c29cf90271280bdcae)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:33.359958 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd8c99e-c394-45af-91cd-75a066de783b None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:33.362264 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 82/147] 10.222.0.81 () {56 vars in 1186 bytes} [Wed Jul 20 22:25:33 2022] GET /identity/v3/projects/da54f6860e1f43c29cf90271280bdcae => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:25:33.367396 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342f767f-6069-4020-91f2-7e59661e8290 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:33.429281 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:33.500596 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342f767f-6069-4020-91f2-7e59661e8290 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:33.542561 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d89bee2-2db6-4da9-b29a-185259f98f1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:33.543746 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d89bee2-2db6-4da9-b29a-185259f98f1f None admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:33.544184 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d89bee2-2db6-4da9-b29a-185259f98f1f None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:33.544857 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d89bee2-2db6-4da9-b29a-185259f98f1f None admin] PATH_INFO: `/v3/projects/da54f6860e1f43c29cf90271280bdcae/users/1d8eed80f6204aa4b05b6f0d3edd0d8b/roles/5b2ef0feadf94c059dffbd7d8bc03aa4` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:33.547119 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d89bee2-2db6-4da9-b29a-185259f98f1f None admin] RBAC: Authorizing `identity:create_grant(project_id=da54f6860e1f43c29cf90271280bdcae, user_id=1d8eed80f6204aa4b05b6f0d3edd0d8b, role_id=5b2ef0feadf94c059dffbd7d8bc03aa4)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:33.548740 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d89bee2-2db6-4da9-b29a-185259f98f1f None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:33.634403 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 66/148] 10.222.0.81 () {58 vars in 1439 bytes} [Wed Jul 20 22:25:33 2022] PUT /identity/v3/projects/da54f6860e1f43c29cf90271280bdcae/users/1d8eed80f6204aa4b05b6f0d3edd0d8b/roles/5b2ef0feadf94c059dffbd7d8bc03aa4 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:25:33.692558 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859cb834-188b-421c-8eb8-f6b867d78061 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:33.693029 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859cb834-188b-421c-8eb8-f6b867d78061 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:33.693407 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859cb834-188b-421c-8eb8-f6b867d78061 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:33.695213 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 83/149] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:25:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:25:33.701976 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd0dba8-e953-47a2-9107-ec2355dc1c23 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:33.702408 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd0dba8-e953-47a2-9107-ec2355dc1c23 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:33.702798 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd0dba8-e953-47a2-9107-ec2355dc1c23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:33.721400 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-ccd0dba8-e953-47a2-9107-ec2355dc1c23 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:25:33.722638 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-ccd0dba8-e953-47a2-9107-ec2355dc1c23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:25:33.807822 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 67/150] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:25:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 22:25:33.834749 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859cb834-188b-421c-8eb8-f6b867d78061 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:33.854986 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:33.958457 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859cb834-188b-421c-8eb8-f6b867d78061 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:33.995805 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a489bf0f-f115-4387-aea1-3ea4072fa86b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:33.996669 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a489bf0f-f115-4387-aea1-3ea4072fa86b None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:33.996961 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a489bf0f-f115-4387-aea1-3ea4072fa86b None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:33.997239 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a489bf0f-f115-4387-aea1-3ea4072fa86b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:34.008521 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a489bf0f-f115-4387-aea1-3ea4072fa86b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 22:25:34.017554 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-a489bf0f-f115-4387-aea1-3ea4072fa86b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 22:25:34.017554 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:25:34.017554 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:25:34.017554 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:25:34.017554 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:25:34.017554 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:25:34.017554 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:25:34.017554 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:25:34.017554 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:25:34.017554 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:25:34.017554 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:25:34.017554 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:25:34.017554 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 22:25:34.017554 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 22:25:34.017554 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 22:25:34.017554 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 22:25:34.018397 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 22:25:34.018397 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 22:25:34.018397 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 22:25:34.018397 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 22:25:34.018397 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:25:34.018397 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:25:34.018397 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:25:34.018397 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:25:34.018397 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:25:34.018397 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:25:34.018397 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:25:34.018397 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:25:34.018397 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:25:34.018397 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:25:34.018397 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:25:34.018397 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:25:34.019617 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:25:34.019617 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:25:34.019617 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:25:34.019617 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:25:34.019617 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 22:25:34.019617 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 22:25:34.019617 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 22:25:34.019617 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 22:25:34.019617 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 22:25:34.019617 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 22:25:34.019617 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 22:25:34.019617 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:25:34.019617 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 84/151] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Jul 20 22:25:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:25:34.025060 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd0dba8-e953-47a2-9107-ec2355dc1c23 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:34.064405 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:34.145283 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd0dba8-e953-47a2-9107-ec2355dc1c23 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:34.183233 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d8f892-39b0-4671-89ab-a4e3f6b8d461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:34.183932 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d8f892-39b0-4671-89ab-a4e3f6b8d461 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:34.184224 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d8f892-39b0-4671-89ab-a4e3f6b8d461 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:34.184511 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d8f892-39b0-4671-89ab-a4e3f6b8d461 None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:34.185317 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d8f892-39b0-4671-89ab-a4e3f6b8d461 None admin] RBAC: Adding query filter params (name=member) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:25:34.185644 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d8f892-39b0-4671-89ab-a4e3f6b8d461 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:34.187181 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d8f892-39b0-4671-89ab-a4e3f6b8d461 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:34.196554 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 68/152] 10.222.0.81 () {56 vars in 1101 bytes} [Wed Jul 20 22:25:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:25:34.202562 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a489bf0f-f115-4387-aea1-3ea4072fa86b None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:34.286288 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:34.313538 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a489bf0f-f115-4387-aea1-3ea4072fa86b None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:34.350367 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd260975-c653-4a2d-a2f2-097fc5c61813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:34.351297 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd260975-c653-4a2d-a2f2-097fc5c61813 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:34.351586 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd260975-c653-4a2d-a2f2-097fc5c61813 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:34.351869 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd260975-c653-4a2d-a2f2-097fc5c61813 None admin] PATH_INFO: `/v3/users/74bf4b54c7664f9fac20de5b75d465f8` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:34.353100 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd260975-c653-4a2d-a2f2-097fc5c61813 None admin] RBAC: Authorizing `identity:get_user(user_id=74bf4b54c7664f9fac20de5b75d465f8)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:34.354605 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd260975-c653-4a2d-a2f2-097fc5c61813 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:34.356538 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 85/153] 10.222.0.81 () {56 vars in 1177 bytes} [Wed Jul 20 22:25:34 2022] GET /identity/v3/users/74bf4b54c7664f9fac20de5b75d465f8 => generated 275 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:25:34.361983 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d8f892-39b0-4671-89ab-a4e3f6b8d461 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:34.466345 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:34.474962 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d8f892-39b0-4671-89ab-a4e3f6b8d461 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:34.514424 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fce2e1-6025-493d-b3c5-aa159160d37d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:34.515199 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fce2e1-6025-493d-b3c5-aa159160d37d None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:34.515494 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fce2e1-6025-493d-b3c5-aa159160d37d None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:34.515830 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fce2e1-6025-493d-b3c5-aa159160d37d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:34.596902 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fce2e1-6025-493d-b3c5-aa159160d37d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:34.598467 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fce2e1-6025-493d-b3c5-aa159160d37d None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:34.600238 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6425e32-59b2-4037-858a-4b50ef444ffa None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:34.600528 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6425e32-59b2-4037-858a-4b50ef444ffa None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:34.600811 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6425e32-59b2-4037-858a-4b50ef444ffa None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:34.602255 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 86/154] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:25:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:25:34.613078 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e739ef41-e746-401a-af07-8fd0a09524f1 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:34.613382 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e739ef41-e746-401a-af07-8fd0a09524f1 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:34.613651 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e739ef41-e746-401a-af07-8fd0a09524f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:34.645518 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-e739ef41-e746-401a-af07-8fd0a09524f1 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:25:34.646754 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-e739ef41-e746-401a-af07-8fd0a09524f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:25:34.757054 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:34.765767 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 69/155] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Jul 20 22:25:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 22:25:34.767454 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e19221-aa09-4d3f-abe8-992eaa134125 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:34.775060 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e19221-aa09-4d3f-abe8-992eaa134125 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:34.775060 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e19221-aa09-4d3f-abe8-992eaa134125 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:34.775060 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 70/156] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:25:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:25:34.776501 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea720040-764c-4870-9c1d-f02db4324ae9 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:34.776867 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea720040-764c-4870-9c1d-f02db4324ae9 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:34.777223 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea720040-764c-4870-9c1d-f02db4324ae9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:34.781913 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 87/157] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:25:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 22:25:34.783032 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e739ef41-e746-401a-af07-8fd0a09524f1 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:34.799064 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-ea720040-764c-4870-9c1d-f02db4324ae9 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:25:34.799064 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-ea720040-764c-4870-9c1d-f02db4324ae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:25:34.887188 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 71/158] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:25:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 22:25:34.887976 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea720040-764c-4870-9c1d-f02db4324ae9 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:34.922281 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:34.981488 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e739ef41-e746-401a-af07-8fd0a09524f1 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:35.008001 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea720040-764c-4870-9c1d-f02db4324ae9 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:35.037067 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:35.044281 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7a0b7a-4c98-4ae4-9ad0-3d9d3933c5b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:35.045632 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7a0b7a-4c98-4ae4-9ad0-3d9d3933c5b1 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:35.046270 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7a0b7a-4c98-4ae4-9ad0-3d9d3933c5b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:35.046701 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7a0b7a-4c98-4ae4-9ad0-3d9d3933c5b1 None admin] PATH_INFO: `/v3/projects/da54f6860e1f43c29cf90271280bdcae` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:35.049384 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecffd89-bd1f-4b6f-8bd3-db3ad9dbdc7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:35.050503 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7a0b7a-4c98-4ae4-9ad0-3d9d3933c5b1 None admin] RBAC: Authorizing `identity:get_project(project_id=da54f6860e1f43c29cf90271280bdcae)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:35.050636 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecffd89-bd1f-4b6f-8bd3-db3ad9dbdc7c None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:35.050940 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecffd89-bd1f-4b6f-8bd3-db3ad9dbdc7c None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:35.051312 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecffd89-bd1f-4b6f-8bd3-db3ad9dbdc7c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:35.052618 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7a0b7a-4c98-4ae4-9ad0-3d9d3933c5b1 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:35.055376 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 88/159] 10.222.0.81 () {56 vars in 1186 bytes} [Wed Jul 20 22:25:34 2022] GET /identity/v3/projects/da54f6860e1f43c29cf90271280bdcae => generated 298 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:25:35.055636 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7a0b7a-4c98-4ae4-9ad0-3d9d3933c5b1 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:35.068150 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecffd89-bd1f-4b6f-8bd3-db3ad9dbdc7c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 22:25:35.080028 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-fecffd89-bd1f-4b6f-8bd3-db3ad9dbdc7c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 22:25:35.080028 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:25:35.080028 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:25:35.080028 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:25:35.080028 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:25:35.080028 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:25:35.080028 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:25:35.080028 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:25:35.080028 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:25:35.080028 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:25:35.080028 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:25:35.080028 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:25:35.080028 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 22:25:35.080028 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 22:25:35.080028 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 22:25:35.080028 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 22:25:35.080946 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 22:25:35.080946 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 22:25:35.080946 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 22:25:35.080946 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 22:25:35.080946 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:25:35.080946 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:25:35.080946 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:25:35.080946 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:25:35.080946 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:25:35.080946 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:25:35.080946 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:25:35.080946 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:25:35.080946 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:25:35.080946 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:25:35.080946 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:25:35.080946 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:25:35.081965 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:25:35.081965 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:25:35.081965 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:25:35.081965 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:25:35.081965 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 22:25:35.081965 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 22:25:35.081965 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 22:25:35.081965 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 22:25:35.081965 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 22:25:35.081965 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 22:25:35.081965 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 22:25:35.081965 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:25:35.081965 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 72/160] 10.222.0.81 () {56 vars in 1114 bytes} [Wed Jul 20 22:25:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:25:35.082610 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecffd89-bd1f-4b6f-8bd3-db3ad9dbdc7c None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:35.231563 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecffd89-bd1f-4b6f-8bd3-db3ad9dbdc7c None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:35.263595 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7a0b7a-4c98-4ae4-9ad0-3d9d3933c5b1 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:35.296322 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba7cd97-3b4d-41ab-ac9e-e47dd26884e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:35.296998 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba7cd97-3b4d-41ab-ac9e-e47dd26884e7 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:35.297288 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba7cd97-3b4d-41ab-ac9e-e47dd26884e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:35.297576 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba7cd97-3b4d-41ab-ac9e-e47dd26884e7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:35.298431 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba7cd97-3b4d-41ab-ac9e-e47dd26884e7 None admin] RBAC: Adding query filter params (role.id=559942c4de1744169f8b5e0404ecde6b, scope.project.id=da54f6860e1f43c29cf90271280bdcae, user.id=74bf4b54c7664f9fac20de5b75d465f8) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:25:35.298749 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba7cd97-3b4d-41ab-ac9e-e47dd26884e7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:35.300485 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba7cd97-3b4d-41ab-ac9e-e47dd26884e7 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:35.312844 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 73/161] 10.222.0.81 () {56 vars in 1374 bytes} [Wed Jul 20 22:25:35 2022] GET /identity/v3/role_assignments?user.id=74bf4b54c7664f9fac20de5b75d465f8&scope.project.id=da54f6860e1f43c29cf90271280bdcae&role.id=559942c4de1744169f8b5e0404ecde6b => generated 608 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:25:35.313687 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba7cd97-3b4d-41ab-ac9e-e47dd26884e7 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:35.350740 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40864582-9d31-4b05-8f93-a1a44e4f1a0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:35.351715 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40864582-9d31-4b05-8f93-a1a44e4f1a0d None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:35.352060 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40864582-9d31-4b05-8f93-a1a44e4f1a0d None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:35.352352 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40864582-9d31-4b05-8f93-a1a44e4f1a0d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:35.368485 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:35.376060 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40864582-9d31-4b05-8f93-a1a44e4f1a0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 22:25:35.384741 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-40864582-9d31-4b05-8f93-a1a44e4f1a0d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 22:25:35.384741 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:25:35.384741 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:25:35.384741 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:25:35.384741 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:25:35.384741 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:25:35.384741 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:25:35.384741 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:25:35.384741 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:25:35.384741 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:25:35.384741 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:25:35.384741 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:25:35.384741 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 22:25:35.384741 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 22:25:35.384741 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 22:25:35.384741 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 22:25:35.391563 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 22:25:35.391563 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 22:25:35.391563 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 22:25:35.391563 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 22:25:35.391563 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:25:35.391563 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:25:35.391563 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:25:35.391563 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:25:35.391563 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:25:35.391563 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:25:35.391563 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:25:35.391563 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:25:35.391563 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:25:35.391563 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:25:35.391563 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:25:35.391563 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:25:35.392342 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:25:35.392342 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:25:35.392342 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:25:35.392342 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:25:35.392342 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 22:25:35.392342 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 22:25:35.392342 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 22:25:35.392342 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 22:25:35.392342 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 22:25:35.392342 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 22:25:35.392342 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 22:25:35.392342 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:25:35.392342 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 89/162] 10.222.0.81 () {56 vars in 1096 bytes} [Wed Jul 20 22:25:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:25:35.394266 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40864582-9d31-4b05-8f93-a1a44e4f1a0d None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:35.410449 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:35.506540 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba7cd97-3b4d-41ab-ac9e-e47dd26884e7 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:35.567757 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63260cbf-457c-49f3-9656-414547aad12c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:35.568463 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63260cbf-457c-49f3-9656-414547aad12c None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:35.568758 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63260cbf-457c-49f3-9656-414547aad12c None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:35.569044 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63260cbf-457c-49f3-9656-414547aad12c None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:35.569817 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63260cbf-457c-49f3-9656-414547aad12c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:25:35.570136 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63260cbf-457c-49f3-9656-414547aad12c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:35.571654 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63260cbf-457c-49f3-9656-414547aad12c None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:35.587596 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 74/163] 10.222.0.81 () {56 vars in 1111 bytes} [Wed Jul 20 22:25:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:25:35.589772 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9760ec31-1d1a-4610-a1b4-113662b94e4f None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:35.590074 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9760ec31-1d1a-4610-a1b4-113662b94e4f None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:35.590357 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9760ec31-1d1a-4610-a1b4-113662b94e4f None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:35.594455 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 75/164] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:25:35 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:25:35.598614 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9760ec31-1d1a-4610-a1b4-113662b94e4f None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:35.615722 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40864582-9d31-4b05-8f93-a1a44e4f1a0d None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:35.680172 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026869c4-9e16-4c8e-8670-3001d2990a90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:35.680859 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026869c4-9e16-4c8e-8670-3001d2990a90 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:35.681151 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026869c4-9e16-4c8e-8670-3001d2990a90 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:35.681437 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026869c4-9e16-4c8e-8670-3001d2990a90 None admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:35.682243 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026869c4-9e16-4c8e-8670-3001d2990a90 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:25:35.682558 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026869c4-9e16-4c8e-8670-3001d2990a90 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:35.683989 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026869c4-9e16-4c8e-8670-3001d2990a90 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:35.704206 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 90/165] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Jul 20 22:25:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 22:25:35.711500 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18707729-c7a0-46d6-8911-1ca0b51ab3f2 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:35.711825 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18707729-c7a0-46d6-8911-1ca0b51ab3f2 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:35.712123 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18707729-c7a0-46d6-8911-1ca0b51ab3f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:35.745269 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-18707729-c7a0-46d6-8911-1ca0b51ab3f2 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:25:35.746776 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-18707729-c7a0-46d6-8911-1ca0b51ab3f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:25:35.816419 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9760ec31-1d1a-4610-a1b4-113662b94e4f None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:35.837771 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:35.856411 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:35.881740 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 91/166] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:25:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 22:25:35.883530 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18707729-c7a0-46d6-8911-1ca0b51ab3f2 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:35.900685 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41227227-7f3b-440c-bb06-f14583ca9d99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:35.901604 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41227227-7f3b-440c-bb06-f14583ca9d99 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:35.901902 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41227227-7f3b-440c-bb06-f14583ca9d99 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:35.902184 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41227227-7f3b-440c-bb06-f14583ca9d99 None admin] PATH_INFO: `/v3/users/74bf4b54c7664f9fac20de5b75d465f8` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:35.907743 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41227227-7f3b-440c-bb06-f14583ca9d99 None admin] RBAC: Authorizing `identity:get_user(user_id=74bf4b54c7664f9fac20de5b75d465f8)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:35.909216 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41227227-7f3b-440c-bb06-f14583ca9d99 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:35.910911 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 76/167] 10.222.0.81 () {56 vars in 1177 bytes} [Wed Jul 20 22:25:35 2022] GET /identity/v3/users/74bf4b54c7664f9fac20de5b75d465f8 => generated 275 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:25:35.918139 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41227227-7f3b-440c-bb06-f14583ca9d99 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:36.083076 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18707729-c7a0-46d6-8911-1ca0b51ab3f2 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:36.133357 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41227227-7f3b-440c-bb06-f14583ca9d99 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:36.157166 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45ca224-e61e-48c1-a887-a4bde1d00efd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:36.158116 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45ca224-e61e-48c1-a887-a4bde1d00efd None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:36.158410 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45ca224-e61e-48c1-a887-a4bde1d00efd None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:36.158693 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45ca224-e61e-48c1-a887-a4bde1d00efd None admin] PATH_INFO: `/v3/users/98f5d4c4be4845478aa0d31a1403d775` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:36.160635 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45ca224-e61e-48c1-a887-a4bde1d00efd None admin] RBAC: Authorizing `identity:get_user(user_id=98f5d4c4be4845478aa0d31a1403d775)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:36.162874 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45ca224-e61e-48c1-a887-a4bde1d00efd None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:36.164852 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 92/168] 10.222.0.81 () {56 vars in 1177 bytes} [Wed Jul 20 22:25:35 2022] GET /identity/v3/users/98f5d4c4be4845478aa0d31a1403d775 => generated 247 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:25:36.165474 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45ca224-e61e-48c1-a887-a4bde1d00efd None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:36.191356 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d774156-3b36-46f8-b686-3eccbadf27c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:36.192010 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d774156-3b36-46f8-b686-3eccbadf27c3 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:36.192301 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d774156-3b36-46f8-b686-3eccbadf27c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:36.192586 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d774156-3b36-46f8-b686-3eccbadf27c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:36.297542 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:36.313154 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:36.332666 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d774156-3b36-46f8-b686-3eccbadf27c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:36.334217 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d774156-3b36-46f8-b686-3eccbadf27c3 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:36.421410 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45ca224-e61e-48c1-a887-a4bde1d00efd None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:36.473526 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 77/169] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Jul 20 22:25:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 556 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 22:25:36.474392 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d774156-3b36-46f8-b686-3eccbadf27c3 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:36.516276 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71148fa-0054-4abd-b768-51d3d8f76356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:36.517110 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71148fa-0054-4abd-b768-51d3d8f76356 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:36.517395 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71148fa-0054-4abd-b768-51d3d8f76356 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:36.517671 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71148fa-0054-4abd-b768-51d3d8f76356 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:36.537941 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71148fa-0054-4abd-b768-51d3d8f76356 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 22:25:36.551112 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-b71148fa-0054-4abd-b768-51d3d8f76356 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 22:25:36.551112 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:25:36.551112 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:25:36.551112 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:25:36.551112 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:25:36.551112 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:25:36.551112 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:25:36.551112 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:25:36.551112 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:25:36.551112 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:25:36.551112 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:25:36.551112 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:25:36.551112 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 22:25:36.551112 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 22:25:36.551112 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 22:25:36.551112 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 22:25:36.553331 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 22:25:36.553331 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 22:25:36.553331 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 22:25:36.553331 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 22:25:36.553331 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:25:36.553331 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:25:36.553331 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:25:36.553331 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:25:36.553331 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:25:36.553331 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:25:36.553331 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:25:36.553331 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:25:36.553331 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:25:36.553331 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:25:36.553331 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:25:36.553331 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:25:36.554147 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:25:36.554147 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:25:36.554147 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:25:36.554147 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:25:36.554147 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 22:25:36.554147 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 22:25:36.554147 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 22:25:36.554147 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 22:25:36.554147 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 22:25:36.554147 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 22:25:36.554147 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 22:25:36.554147 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:25:36.554147 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 93/170] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Jul 20 22:25:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 387 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:25:36.554147 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71148fa-0054-4abd-b768-51d3d8f76356 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:36.557952 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:36.590417 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d774156-3b36-46f8-b686-3eccbadf27c3 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:36.635961 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9a8955-cc9f-4dce-aaa9-d6f4cc680bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:36.636629 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9a8955-cc9f-4dce-aaa9-d6f4cc680bdf None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:36.636917 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9a8955-cc9f-4dce-aaa9-d6f4cc680bdf None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:36.637195 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9a8955-cc9f-4dce-aaa9-d6f4cc680bdf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:36.738894 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9a8955-cc9f-4dce-aaa9-d6f4cc680bdf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:36.740497 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9a8955-cc9f-4dce-aaa9-d6f4cc680bdf None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:36.747697 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71148fa-0054-4abd-b768-51d3d8f76356 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:36.784875 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:36.809638 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a782e7-dd8f-4542-b425-394dc74dd1c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:36.810938 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a782e7-dd8f-4542-b425-394dc74dd1c3 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:36.811420 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a782e7-dd8f-4542-b425-394dc74dd1c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:36.811864 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a782e7-dd8f-4542-b425-394dc74dd1c3 None admin] PATH_INFO: `/v3/projects/da54f6860e1f43c29cf90271280bdcae` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:36.815197 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a782e7-dd8f-4542-b425-394dc74dd1c3 None admin] RBAC: Authorizing `identity:get_project(project_id=da54f6860e1f43c29cf90271280bdcae)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:36.816876 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a782e7-dd8f-4542-b425-394dc74dd1c3 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:36.818991 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 94/171] 10.222.0.81 () {56 vars in 1186 bytes} [Wed Jul 20 22:25:36 2022] GET /identity/v3/projects/da54f6860e1f43c29cf90271280bdcae => generated 298 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:25:36.820086 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a782e7-dd8f-4542-b425-394dc74dd1c3 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:36.826291 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:36.850812 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 78/172] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Jul 20 22:25:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 22:25:36.851621 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9a8955-cc9f-4dce-aaa9-d6f4cc680bdf None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:36.962039 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9a8955-cc9f-4dce-aaa9-d6f4cc680bdf None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:37.005334 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10175db-fb40-4137-83bd-8f3441c05781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:37.006017 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10175db-fb40-4137-83bd-8f3441c05781 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:37.006307 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10175db-fb40-4137-83bd-8f3441c05781 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:37.006589 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10175db-fb40-4137-83bd-8f3441c05781 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:37.007457 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10175db-fb40-4137-83bd-8f3441c05781 None admin] RBAC: Adding query filter params (role.id=6d5a3e6a22fa4f4fa47afcfce0246a1f, scope.project.id=da54f6860e1f43c29cf90271280bdcae, user.id=74bf4b54c7664f9fac20de5b75d465f8) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:25:37.007777 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10175db-fb40-4137-83bd-8f3441c05781 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:37.009237 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10175db-fb40-4137-83bd-8f3441c05781 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:37.022499 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 79/173] 10.222.0.81 () {56 vars in 1374 bytes} [Wed Jul 20 22:25:36 2022] GET /identity/v3/role_assignments?user.id=74bf4b54c7664f9fac20de5b75d465f8&scope.project.id=da54f6860e1f43c29cf90271280bdcae&role.id=6d5a3e6a22fa4f4fa47afcfce0246a1f => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:25:37.023371 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10175db-fb40-4137-83bd-8f3441c05781 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:37.026295 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a782e7-dd8f-4542-b425-394dc74dd1c3 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:37.126532 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cf1fb2-16e0-4fff-9fa7-82251a5a3a67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:37.127331 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cf1fb2-16e0-4fff-9fa7-82251a5a3a67 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:37.127555 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cf1fb2-16e0-4fff-9fa7-82251a5a3a67 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:37.127819 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cf1fb2-16e0-4fff-9fa7-82251a5a3a67 None admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:37.128598 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cf1fb2-16e0-4fff-9fa7-82251a5a3a67 None admin] RBAC: Adding query filter params (name=member) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:25:37.128913 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cf1fb2-16e0-4fff-9fa7-82251a5a3a67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:37.130337 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cf1fb2-16e0-4fff-9fa7-82251a5a3a67 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:37.137629 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:37.155344 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 95/174] 10.222.0.81 () {56 vars in 1101 bytes} [Wed Jul 20 22:25:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:25:37.156610 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5064b5-b46d-444a-9ea9-88605a03f048 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:37.157680 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5064b5-b46d-444a-9ea9-88605a03f048 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:37.157680 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5064b5-b46d-444a-9ea9-88605a03f048 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:37.158424 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 96/175] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:25:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:25:37.167081 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6f2bc5-f8cd-428d-bbfc-c63f58c26f35 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:37.167391 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6f2bc5-f8cd-428d-bbfc-c63f58c26f35 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:37.167658 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6f2bc5-f8cd-428d-bbfc-c63f58c26f35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:37.220093 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-1c6f2bc5-f8cd-428d-bbfc-c63f58c26f35 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:25:37.221320 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-1c6f2bc5-f8cd-428d-bbfc-c63f58c26f35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:25:37.271575 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10175db-fb40-4137-83bd-8f3441c05781 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:37.296793 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:37.326480 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 97/176] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:25:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 22:25:37.326480 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6f2bc5-f8cd-428d-bbfc-c63f58c26f35 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:37.343933 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0de9c3-b02d-494e-9023-fda4267c603b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:37.344833 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0de9c3-b02d-494e-9023-fda4267c603b None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:37.345119 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0de9c3-b02d-494e-9023-fda4267c603b None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:37.345395 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0de9c3-b02d-494e-9023-fda4267c603b None admin] PATH_INFO: `/v3/projects/da54f6860e1f43c29cf90271280bdcae` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:37.348837 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0de9c3-b02d-494e-9023-fda4267c603b None admin] RBAC: Authorizing `identity:get_project(project_id=da54f6860e1f43c29cf90271280bdcae)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:37.350311 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0de9c3-b02d-494e-9023-fda4267c603b None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:37.352528 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 80/177] 10.222.0.81 () {56 vars in 1186 bytes} [Wed Jul 20 22:25:37 2022] GET /identity/v3/projects/da54f6860e1f43c29cf90271280bdcae => generated 298 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:25:37.359877 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0de9c3-b02d-494e-9023-fda4267c603b None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:37.500551 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6f2bc5-f8cd-428d-bbfc-c63f58c26f35 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:37.512935 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0de9c3-b02d-494e-9023-fda4267c603b None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:37.524613 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:37.535122 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9c8e7a-be2a-4507-afce-6a6a98fc224c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:37.538291 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9c8e7a-be2a-4507-afce-6a6a98fc224c None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:37.538291 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9c8e7a-be2a-4507-afce-6a6a98fc224c None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:37.538291 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9c8e7a-be2a-4507-afce-6a6a98fc224c None admin] PATH_INFO: `/v3/users/74bf4b54c7664f9fac20de5b75d465f8` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:37.538291 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9c8e7a-be2a-4507-afce-6a6a98fc224c None admin] RBAC: Authorizing `identity:get_user(user_id=74bf4b54c7664f9fac20de5b75d465f8)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:37.539208 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9c8e7a-be2a-4507-afce-6a6a98fc224c None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:37.544906 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 98/178] 10.222.0.81 () {56 vars in 1177 bytes} [Wed Jul 20 22:25:37 2022] GET /identity/v3/users/74bf4b54c7664f9fac20de5b75d465f8 => generated 275 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:25:37.544906 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9c8e7a-be2a-4507-afce-6a6a98fc224c None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:37.555365 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a71e5a1-ad8f-4d13-9325-bd39f87579dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:37.556045 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a71e5a1-ad8f-4d13-9325-bd39f87579dd None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:37.556336 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a71e5a1-ad8f-4d13-9325-bd39f87579dd None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:37.557200 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a71e5a1-ad8f-4d13-9325-bd39f87579dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:37.558099 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a71e5a1-ad8f-4d13-9325-bd39f87579dd None admin] RBAC: Adding query filter params (role.id=d89b6c92d35a401a8517f1d47f6be843, scope.project.id=da54f6860e1f43c29cf90271280bdcae, user.id=98f5d4c4be4845478aa0d31a1403d775) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:25:37.558419 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a71e5a1-ad8f-4d13-9325-bd39f87579dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:37.559908 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a71e5a1-ad8f-4d13-9325-bd39f87579dd None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:37.571051 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 81/179] 10.222.0.81 () {56 vars in 1374 bytes} [Wed Jul 20 22:25:37 2022] GET /identity/v3/role_assignments?user.id=98f5d4c4be4845478aa0d31a1403d775&scope.project.id=da54f6860e1f43c29cf90271280bdcae&role.id=d89b6c92d35a401a8517f1d47f6be843 => generated 608 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:25:37.571051 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baec217-c51c-409d-9a6d-8f21fdba7e1b None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:37.571051 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baec217-c51c-409d-9a6d-8f21fdba7e1b None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:37.571051 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baec217-c51c-409d-9a6d-8f21fdba7e1b None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:37.571051 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 82/180] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:25:37 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:25:37.571051 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db429b2a-d04e-4c96-a281-fa359ec2f467 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:37.571527 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db429b2a-d04e-4c96-a281-fa359ec2f467 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:37.571527 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db429b2a-d04e-4c96-a281-fa359ec2f467 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:37.575061 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 83/181] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:25:37 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:25:37.575061 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db429b2a-d04e-4c96-a281-fa359ec2f467 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:37.614657 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:37.685443 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9c8e7a-be2a-4507-afce-6a6a98fc224c None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:37.700789 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db429b2a-d04e-4c96-a281-fa359ec2f467 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:37.726565 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd0a65b-66f3-43de-a6a8-0373a4da2dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:37.727335 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd0a65b-66f3-43de-a6a8-0373a4da2dd3 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:37.727685 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd0a65b-66f3-43de-a6a8-0373a4da2dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:37.727970 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd0a65b-66f3-43de-a6a8-0373a4da2dd3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:37.739295 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d44d6ed-f391-4d68-b4a7-e3b08d1224b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:37.739966 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d44d6ed-f391-4d68-b4a7-e3b08d1224b3 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:37.740258 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d44d6ed-f391-4d68-b4a7-e3b08d1224b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:37.740358 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd0a65b-66f3-43de-a6a8-0373a4da2dd3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 22:25:37.740545 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d44d6ed-f391-4d68-b4a7-e3b08d1224b3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:37.749255 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-8fd0a65b-66f3-43de-a6a8-0373a4da2dd3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 22:25:37.749255 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:25:37.749255 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:25:37.749255 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:25:37.749255 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:25:37.749255 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:25:37.749255 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:25:37.749255 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:25:37.749255 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:25:37.749255 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:25:37.749255 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:25:37.749255 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:25:37.749255 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 22:25:37.749255 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 22:25:37.749255 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 22:25:37.749255 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 22:25:37.750157 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 22:25:37.750157 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 22:25:37.750157 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 22:25:37.750157 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 22:25:37.750157 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:25:37.750157 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:25:37.750157 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:25:37.750157 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:25:37.750157 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:25:37.750157 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:25:37.750157 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:25:37.750157 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:25:37.750157 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:25:37.750157 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:25:37.750157 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:25:37.750157 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:25:37.750985 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:25:37.750985 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:25:37.750985 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:25:37.750985 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:25:37.750985 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 22:25:37.750985 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 22:25:37.750985 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 22:25:37.750985 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 22:25:37.750985 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 22:25:37.750985 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 22:25:37.750985 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 22:25:37.750985 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:25:37.750985 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 99/182] 10.222.0.81 () {56 vars in 1096 bytes} [Wed Jul 20 22:25:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:25:37.752267 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bbaefc-04df-4d58-88c7-2324456428e8 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:37.752267 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bbaefc-04df-4d58-88c7-2324456428e8 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:37.752429 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bbaefc-04df-4d58-88c7-2324456428e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:37.769907 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-a7bbaefc-04df-4d58-88c7-2324456428e8 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:25:37.771530 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-a7bbaefc-04df-4d58-88c7-2324456428e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:25:37.793398 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:37.820960 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d44d6ed-f391-4d68-b4a7-e3b08d1224b3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:37.822456 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d44d6ed-f391-4d68-b4a7-e3b08d1224b3 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:37.845739 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 100/183] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:25:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 22:25:37.847115 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06778a07-d4f0-4e25-8ccd-5581bf45cc6b None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:37.847422 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06778a07-d4f0-4e25-8ccd-5581bf45cc6b None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:37.847624 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06778a07-d4f0-4e25-8ccd-5581bf45cc6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:37.871321 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-06778a07-d4f0-4e25-8ccd-5581bf45cc6b None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:25:37.872519 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-06778a07-d4f0-4e25-8ccd-5581bf45cc6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:25:37.903602 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:37.921891 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 84/184] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Jul 20 22:25:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 22:25:37.922689 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d44d6ed-f391-4d68-b4a7-e3b08d1224b3 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:37.940130 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 101/185] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:25:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 22:25:37.947066 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8a5ecb-24f4-4aac-ac35-05c9f4d36e6f None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:37.948012 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8a5ecb-24f4-4aac-ac35-05c9f4d36e6f None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:37.948300 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8a5ecb-24f4-4aac-ac35-05c9f4d36e6f None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:37.953757 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 102/186] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:25:37 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:25:37.954538 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8a5ecb-24f4-4aac-ac35-05c9f4d36e6f None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:37.999316 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:38.024300 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d44d6ed-f391-4d68-b4a7-e3b08d1224b3 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:38.054975 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8a5ecb-24f4-4aac-ac35-05c9f4d36e6f None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:38.057868 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fe37b9-dcf5-4f33-b104-44364d948d0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:38.058542 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fe37b9-dcf5-4f33-b104-44364d948d0c None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:38.058829 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fe37b9-dcf5-4f33-b104-44364d948d0c None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:38.059138 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fe37b9-dcf5-4f33-b104-44364d948d0c None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:38.059880 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fe37b9-dcf5-4f33-b104-44364d948d0c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:25:38.060191 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fe37b9-dcf5-4f33-b104-44364d948d0c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:38.061599 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fe37b9-dcf5-4f33-b104-44364d948d0c None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:38.069531 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 85/187] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Jul 20 22:25:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:25:38.071900 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fe37b9-dcf5-4f33-b104-44364d948d0c None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:38.090776 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff74465f-b781-4103-a8e4-a0288d1596a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:38.091616 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff74465f-b781-4103-a8e4-a0288d1596a5 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:38.091907 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff74465f-b781-4103-a8e4-a0288d1596a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:38.092188 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff74465f-b781-4103-a8e4-a0288d1596a5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:38.102327 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff74465f-b781-4103-a8e4-a0288d1596a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 22:25:38.110604 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-ff74465f-b781-4103-a8e4-a0288d1596a5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 22:25:38.110604 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:25:38.110604 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:25:38.110604 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:25:38.110604 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:25:38.110604 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:25:38.110604 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:25:38.110604 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:25:38.110604 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:25:38.110604 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:25:38.110604 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:25:38.110604 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:25:38.110604 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 22:25:38.110604 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 22:25:38.110604 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 22:25:38.110604 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 22:25:38.111637 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 22:25:38.111637 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 22:25:38.111637 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 22:25:38.111637 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 22:25:38.111637 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:25:38.111637 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:25:38.111637 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:25:38.111637 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:25:38.111637 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:25:38.111637 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:25:38.111637 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:25:38.111637 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:25:38.111637 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:25:38.111637 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:25:38.111637 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:25:38.111637 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:25:38.112409 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:25:38.112409 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:25:38.112409 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:25:38.112409 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:25:38.112409 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 22:25:38.112409 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 22:25:38.112409 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 22:25:38.112409 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 22:25:38.112409 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 22:25:38.112409 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 22:25:38.112409 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 22:25:38.112409 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:25:38.112409 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 103/188] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Jul 20 22:25:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:25:38.113053 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a71dce-0683-4264-b875-fe754e2db741 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:38.113263 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a71dce-0683-4264-b875-fe754e2db741 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:38.113529 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a71dce-0683-4264-b875-fe754e2db741 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:38.130002 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-f9a71dce-0683-4264-b875-fe754e2db741 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:25:38.131241 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-f9a71dce-0683-4264-b875-fe754e2db741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:25:38.154071 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:38.176243 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fe37b9-dcf5-4f33-b104-44364d948d0c None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:38.199945 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 104/189] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:25:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 22:25:38.200795 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a71dce-0683-4264-b875-fe754e2db741 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:38.218666 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7232d8-12e9-454a-a449-d6c37479a664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:38.219594 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7232d8-12e9-454a-a449-d6c37479a664 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:38.219880 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7232d8-12e9-454a-a449-d6c37479a664 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:38.220160 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7232d8-12e9-454a-a449-d6c37479a664 None admin] PATH_INFO: `/v3/projects/311b0d7cef0c483396ad292837f6d314` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:38.221363 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7232d8-12e9-454a-a449-d6c37479a664 None admin] RBAC: Authorizing `identity:get_project(project_id=311b0d7cef0c483396ad292837f6d314)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:38.222827 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7232d8-12e9-454a-a449-d6c37479a664 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:38.229908 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 86/190] 10.222.0.81 () {56 vars in 1186 bytes} [Wed Jul 20 22:25:38 2022] GET /identity/v3/projects/311b0d7cef0c483396ad292837f6d314 => generated 312 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:25:38.229908 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7232d8-12e9-454a-a449-d6c37479a664 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:38.231500 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:38.299205 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a71dce-0683-4264-b875-fe754e2db741 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:38.326207 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7232d8-12e9-454a-a449-d6c37479a664 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:38.332384 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8067e4b-cdea-4c6e-b28b-feaa8186d471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:38.333192 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8067e4b-cdea-4c6e-b28b-feaa8186d471 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:38.333479 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8067e4b-cdea-4c6e-b28b-feaa8186d471 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:38.333755 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8067e4b-cdea-4c6e-b28b-feaa8186d471 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:38.334915 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8067e4b-cdea-4c6e-b28b-feaa8186d471 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:38.336397 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8067e4b-cdea-4c6e-b28b-feaa8186d471 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:38.338514 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 105/191] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Jul 20 22:25:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:25:38.339299 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8067e4b-cdea-4c6e-b28b-feaa8186d471 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:38.364157 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42058a5e-bd5d-4869-8b5f-073d91d265ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:38.365051 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42058a5e-bd5d-4869-8b5f-073d91d265ed None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:38.365336 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42058a5e-bd5d-4869-8b5f-073d91d265ed None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:38.365611 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42058a5e-bd5d-4869-8b5f-073d91d265ed None admin] PATH_INFO: `/v3/users/89ea290b3b2749e492283b8f12aa876e` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:38.367646 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:38.380310 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42058a5e-bd5d-4869-8b5f-073d91d265ed None admin] RBAC: Authorizing `identity:get_user(user_id=89ea290b3b2749e492283b8f12aa876e)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:38.381761 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42058a5e-bd5d-4869-8b5f-073d91d265ed None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:38.383480 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 87/192] 10.222.0.81 () {56 vars in 1177 bytes} [Wed Jul 20 22:25:38 2022] GET /identity/v3/users/89ea290b3b2749e492283b8f12aa876e => generated 283 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:25:38.383480 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42058a5e-bd5d-4869-8b5f-073d91d265ed None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:38.443667 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:38.445542 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8067e4b-cdea-4c6e-b28b-feaa8186d471 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:38.480900 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729332a9-d7df-455e-8dec-cc5de410fc04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:38.481582 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729332a9-d7df-455e-8dec-cc5de410fc04 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:38.481870 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729332a9-d7df-455e-8dec-cc5de410fc04 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:38.482152 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729332a9-d7df-455e-8dec-cc5de410fc04 None admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:38.482906 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729332a9-d7df-455e-8dec-cc5de410fc04 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:25:38.483255 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729332a9-d7df-455e-8dec-cc5de410fc04 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:38.483830 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42058a5e-bd5d-4869-8b5f-073d91d265ed None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:38.484652 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729332a9-d7df-455e-8dec-cc5de410fc04 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:38.492812 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 106/193] 10.222.0.81 () {56 vars in 1101 bytes} [Wed Jul 20 22:25:38 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:25:38.493798 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729332a9-d7df-455e-8dec-cc5de410fc04 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:38.518131 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccae2ac-3b9f-4d2d-ab81-d83a48738785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:38.518959 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccae2ac-3b9f-4d2d-ab81-d83a48738785 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:38.519276 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccae2ac-3b9f-4d2d-ab81-d83a48738785 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:38.519554 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccae2ac-3b9f-4d2d-ab81-d83a48738785 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:38.529987 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccae2ac-3b9f-4d2d-ab81-d83a48738785 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 22:25:38.538666 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-8ccae2ac-3b9f-4d2d-ab81-d83a48738785 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 22:25:38.538666 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:25:38.538666 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:25:38.538666 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:25:38.538666 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:25:38.538666 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:25:38.538666 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:25:38.538666 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:25:38.538666 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:25:38.538666 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:25:38.538666 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:25:38.538666 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:25:38.538666 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 22:25:38.538666 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 22:25:38.538666 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 22:25:38.538666 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 22:25:38.539696 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 22:25:38.539696 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 22:25:38.539696 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 22:25:38.539696 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 22:25:38.539696 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:25:38.539696 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:25:38.539696 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:25:38.539696 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:25:38.539696 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:25:38.539696 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:25:38.539696 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:25:38.539696 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:25:38.539696 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:25:38.539696 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:25:38.539696 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:25:38.539696 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:25:38.540486 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:25:38.540486 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:25:38.540486 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:25:38.540486 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:25:38.540486 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 22:25:38.540486 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 22:25:38.540486 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 22:25:38.540486 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 22:25:38.540486 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 22:25:38.540486 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 22:25:38.540486 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 22:25:38.540486 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:25:38.540486 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 88/194] 10.222.0.81 () {56 vars in 1114 bytes} [Wed Jul 20 22:25:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:25:38.542124 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccae2ac-3b9f-4d2d-ab81-d83a48738785 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:38.587774 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:38.599738 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729332a9-d7df-455e-8dec-cc5de410fc04 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:38.634457 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ca4415-4ce1-488d-b2d2-9a12cb41a035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:38.635147 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ca4415-4ce1-488d-b2d2-9a12cb41a035 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:38.635471 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ca4415-4ce1-488d-b2d2-9a12cb41a035 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:38.635756 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ca4415-4ce1-488d-b2d2-9a12cb41a035 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:38.636594 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ca4415-4ce1-488d-b2d2-9a12cb41a035 None admin] RBAC: Adding query filter params (role.id=559942c4de1744169f8b5e0404ecde6b, scope.project.id=311b0d7cef0c483396ad292837f6d314, user.id=74bf4b54c7664f9fac20de5b75d465f8) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:25:38.636910 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ca4415-4ce1-488d-b2d2-9a12cb41a035 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:38.638373 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ca4415-4ce1-488d-b2d2-9a12cb41a035 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:38.640061 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccae2ac-3b9f-4d2d-ab81-d83a48738785 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:38.644248 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 107/195] 10.222.0.81 () {56 vars in 1374 bytes} [Wed Jul 20 22:25:38 2022] GET /identity/v3/role_assignments?user.id=74bf4b54c7664f9fac20de5b75d465f8&scope.project.id=311b0d7cef0c483396ad292837f6d314&role.id=559942c4de1744169f8b5e0404ecde6b => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:25:38.645043 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ca4415-4ce1-488d-b2d2-9a12cb41a035 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:38.678952 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:38.693118 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a073b4-442f-47e4-9e86-b2f3fd3866d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:38.693828 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a073b4-442f-47e4-9e86-b2f3fd3866d9 None admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:38.694117 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a073b4-442f-47e4-9e86-b2f3fd3866d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:38.695569 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a073b4-442f-47e4-9e86-b2f3fd3866d9 None admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:38.695569 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a073b4-442f-47e4-9e86-b2f3fd3866d9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:38.696756 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a073b4-442f-47e4-9e86-b2f3fd3866d9 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:38.760574 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ca4415-4ce1-488d-b2d2-9a12cb41a035 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:38.798651 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f378cee-a7c5-4329-90eb-a95992d141d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:38.799448 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f378cee-a7c5-4329-90eb-a95992d141d6 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:38.799878 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f378cee-a7c5-4329-90eb-a95992d141d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:38.800229 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f378cee-a7c5-4329-90eb-a95992d141d6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:38.807251 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 89/196] 10.222.0.81 () {60 vars in 1132 bytes} [Wed Jul 20 22:25:38 2022] POST /identity/v3/users => generated 297 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:25:38.810049 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a073b4-442f-47e4-9e86-b2f3fd3866d9 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:38.828895 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:38.899721 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f378cee-a7c5-4329-90eb-a95992d141d6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:38.902143 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f378cee-a7c5-4329-90eb-a95992d141d6 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:38.944417 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a073b4-442f-47e4-9e86-b2f3fd3866d9 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:38.985060 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbdfe07-cd3d-48a8-a3ec-2ec6bec711dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:38.985625 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbdfe07-cd3d-48a8-a3ec-2ec6bec711dd None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:38.985916 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbdfe07-cd3d-48a8-a3ec-2ec6bec711dd None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:38.986202 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbdfe07-cd3d-48a8-a3ec-2ec6bec711dd None admin] PATH_INFO: `/v3/users/1d8eed80f6204aa4b05b6f0d3edd0d8b` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:38.988357 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbdfe07-cd3d-48a8-a3ec-2ec6bec711dd None admin] RBAC: Authorizing `identity:get_user(user_id=1d8eed80f6204aa4b05b6f0d3edd0d8b)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:38.989832 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbdfe07-cd3d-48a8-a3ec-2ec6bec711dd None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:38.991717 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 90/197] 10.222.0.81 () {56 vars in 1177 bytes} [Wed Jul 20 22:25:38 2022] GET /identity/v3/users/1d8eed80f6204aa4b05b6f0d3edd0d8b => generated 289 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:25:38.992368 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbdfe07-cd3d-48a8-a3ec-2ec6bec711dd None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:39.024710 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 108/198] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Jul 20 22:25:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 22:25:39.025512 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f378cee-a7c5-4329-90eb-a95992d141d6 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:39.048393 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:39.087138 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:39.124234 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbdfe07-cd3d-48a8-a3ec-2ec6bec711dd None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:39.159338 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f378cee-a7c5-4329-90eb-a95992d141d6 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:39.161875 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977ad48e-cc26-4b36-9f54-9316975ef3fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:39.162568 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977ad48e-cc26-4b36-9f54-9316975ef3fc None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:39.162859 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977ad48e-cc26-4b36-9f54-9316975ef3fc None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:39.163172 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977ad48e-cc26-4b36-9f54-9316975ef3fc None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:39.164660 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977ad48e-cc26-4b36-9f54-9316975ef3fc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:25:39.164660 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977ad48e-cc26-4b36-9f54-9316975ef3fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:39.165604 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977ad48e-cc26-4b36-9f54-9316975ef3fc None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:39.175857 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 91/199] 10.222.0.81 () {56 vars in 1111 bytes} [Wed Jul 20 22:25:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:25:39.176619 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977ad48e-cc26-4b36-9f54-9316975ef3fc None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:39.208503 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d831ba91-2113-41a3-a8d3-016d8a5b05aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:39.209169 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d831ba91-2113-41a3-a8d3-016d8a5b05aa None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:39.209456 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d831ba91-2113-41a3-a8d3-016d8a5b05aa None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:39.209735 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d831ba91-2113-41a3-a8d3-016d8a5b05aa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:39.285276 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d831ba91-2113-41a3-a8d3-016d8a5b05aa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:39.286719 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d831ba91-2113-41a3-a8d3-016d8a5b05aa None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:39.319641 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977ad48e-cc26-4b36-9f54-9316975ef3fc None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:39.332711 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:39.342876 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:39.365865 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e44bac-81a1-4b42-894f-8240f9d69a6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:39.366767 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e44bac-81a1-4b42-894f-8240f9d69a6e None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:39.367081 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e44bac-81a1-4b42-894f-8240f9d69a6e None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:39.367361 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e44bac-81a1-4b42-894f-8240f9d69a6e None admin] PATH_INFO: `/v3/projects/7165d638ee164d27903602ef884fcf03` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:39.368686 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e44bac-81a1-4b42-894f-8240f9d69a6e None admin] RBAC: Authorizing `identity:get_project(project_id=7165d638ee164d27903602ef884fcf03)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:39.370188 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e44bac-81a1-4b42-894f-8240f9d69a6e None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:39.371957 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 92/200] 10.222.0.81 () {56 vars in 1186 bytes} [Wed Jul 20 22:25:39 2022] GET /identity/v3/projects/7165d638ee164d27903602ef884fcf03 => generated 302 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:25:39.372735 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e44bac-81a1-4b42-894f-8240f9d69a6e None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:39.410117 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 109/201] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Jul 20 22:25:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 22:25:39.411340 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d831ba91-2113-41a3-a8d3-016d8a5b05aa None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:39.497977 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e44bac-81a1-4b42-894f-8240f9d69a6e None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:39.533923 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d831ba91-2113-41a3-a8d3-016d8a5b05aa None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:39.535436 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f46f37-079b-42f4-87ae-af8c9bf4e4f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:39.536348 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f46f37-079b-42f4-87ae-af8c9bf4e4f9 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:39.536637 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f46f37-079b-42f4-87ae-af8c9bf4e4f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:39.536919 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f46f37-079b-42f4-87ae-af8c9bf4e4f9 None admin] PATH_INFO: `/v3/users/89ea290b3b2749e492283b8f12aa876e` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:39.538147 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f46f37-079b-42f4-87ae-af8c9bf4e4f9 None admin] RBAC: Authorizing `identity:get_user(user_id=89ea290b3b2749e492283b8f12aa876e)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:39.539649 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f46f37-079b-42f4-87ae-af8c9bf4e4f9 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:39.541890 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 93/202] 10.222.0.81 () {56 vars in 1177 bytes} [Wed Jul 20 22:25:39 2022] GET /identity/v3/users/89ea290b3b2749e492283b8f12aa876e => generated 283 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:25:39.542675 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f46f37-079b-42f4-87ae-af8c9bf4e4f9 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:39.584258 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b7b902-0cd1-4e2a-ae3d-2c187560624a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:39.585498 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b7b902-0cd1-4e2a-ae3d-2c187560624a None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:39.585788 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b7b902-0cd1-4e2a-ae3d-2c187560624a None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:39.586069 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b7b902-0cd1-4e2a-ae3d-2c187560624a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:39.587395 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b7b902-0cd1-4e2a-ae3d-2c187560624a None admin] RBAC: Adding query filter params (role.id=d89b6c92d35a401a8517f1d47f6be843, scope.project.id=7165d638ee164d27903602ef884fcf03, user.id=89ea290b3b2749e492283b8f12aa876e) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:25:39.587756 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b7b902-0cd1-4e2a-ae3d-2c187560624a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:39.589910 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b7b902-0cd1-4e2a-ae3d-2c187560624a None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:39.591321 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:39.598221 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 110/203] 10.222.0.81 () {56 vars in 1374 bytes} [Wed Jul 20 22:25:39 2022] GET /identity/v3/role_assignments?user.id=89ea290b3b2749e492283b8f12aa876e&scope.project.id=7165d638ee164d27903602ef884fcf03&role.id=d89b6c92d35a401a8517f1d47f6be843 => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:25:39.599051 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b7b902-0cd1-4e2a-ae3d-2c187560624a None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:39.624377 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:39.653956 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f46f37-079b-42f4-87ae-af8c9bf4e4f9 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:39.701530 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746a8134-c6ad-4855-ad3b-55ff17628b2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:39.702365 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746a8134-c6ad-4855-ad3b-55ff17628b2f None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:39.702657 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746a8134-c6ad-4855-ad3b-55ff17628b2f None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:39.702941 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746a8134-c6ad-4855-ad3b-55ff17628b2f None admin] PATH_INFO: `/v3/projects/da54f6860e1f43c29cf90271280bdcae` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:39.705950 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746a8134-c6ad-4855-ad3b-55ff17628b2f None admin] RBAC: Authorizing `identity:get_project(project_id=da54f6860e1f43c29cf90271280bdcae)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:39.707644 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746a8134-c6ad-4855-ad3b-55ff17628b2f None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:39.711142 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 94/204] 10.222.0.81 () {56 vars in 1186 bytes} [Wed Jul 20 22:25:39 2022] GET /identity/v3/projects/da54f6860e1f43c29cf90271280bdcae => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:25:39.717534 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746a8134-c6ad-4855-ad3b-55ff17628b2f None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:39.740229 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b7b902-0cd1-4e2a-ae3d-2c187560624a None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:39.780198 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241e9299-9922-4ab4-9bc8-b0f48825bd18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:39.787354 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241e9299-9922-4ab4-9bc8-b0f48825bd18 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:39.787354 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241e9299-9922-4ab4-9bc8-b0f48825bd18 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:39.787354 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241e9299-9922-4ab4-9bc8-b0f48825bd18 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:39.864899 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241e9299-9922-4ab4-9bc8-b0f48825bd18 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:39.866664 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241e9299-9922-4ab4-9bc8-b0f48825bd18 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:39.887516 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:39.984601 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 111/205] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Jul 20 22:25:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 22:25:39.985687 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241e9299-9922-4ab4-9bc8-b0f48825bd18 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:40.003180 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746a8134-c6ad-4855-ad3b-55ff17628b2f None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:40.027354 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:40.068177 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5491189-d67b-40f2-ad00-4ad21adc3461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:40.068873 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5491189-d67b-40f2-ad00-4ad21adc3461 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:40.069149 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5491189-d67b-40f2-ad00-4ad21adc3461 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:40.069434 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5491189-d67b-40f2-ad00-4ad21adc3461 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:40.070277 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5491189-d67b-40f2-ad00-4ad21adc3461 None admin] RBAC: Adding query filter params (role.id=5b2ef0feadf94c059dffbd7d8bc03aa4, scope.project.id=da54f6860e1f43c29cf90271280bdcae, user.id=1d8eed80f6204aa4b05b6f0d3edd0d8b) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:25:40.070595 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5491189-d67b-40f2-ad00-4ad21adc3461 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:40.072890 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5491189-d67b-40f2-ad00-4ad21adc3461 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:40.080264 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 95/206] 10.222.0.81 () {56 vars in 1374 bytes} [Wed Jul 20 22:25:39 2022] GET /identity/v3/role_assignments?user.id=1d8eed80f6204aa4b05b6f0d3edd0d8b&scope.project.id=da54f6860e1f43c29cf90271280bdcae&role.id=5b2ef0feadf94c059dffbd7d8bc03aa4 => generated 608 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 22:25:40.187039 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241e9299-9922-4ab4-9bc8-b0f48825bd18 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:40.213862 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:40.239000 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa0f285-3b52-41ad-9c76-cee319953ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:40.239839 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa0f285-3b52-41ad-9c76-cee319953ca1 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:40.240122 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa0f285-3b52-41ad-9c76-cee319953ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:40.240400 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa0f285-3b52-41ad-9c76-cee319953ca1 None admin] PATH_INFO: `/v3/projects/7165d638ee164d27903602ef884fcf03` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:40.241600 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa0f285-3b52-41ad-9c76-cee319953ca1 None admin] RBAC: Authorizing `identity:get_project(project_id=7165d638ee164d27903602ef884fcf03)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:40.243097 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa0f285-3b52-41ad-9c76-cee319953ca1 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:40.245670 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 112/207] 10.222.0.81 () {56 vars in 1186 bytes} [Wed Jul 20 22:25:39 2022] GET /identity/v3/projects/7165d638ee164d27903602ef884fcf03 => generated 302 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 22:25:40.251430 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5491189-d67b-40f2-ad00-4ad21adc3461 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:40.389592 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5491189-d67b-40f2-ad00-4ad21adc3461 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:40.424433 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a49bfa2-f52c-40c4-9019-481b151d4aa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:40.425458 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a49bfa2-f52c-40c4-9019-481b151d4aa8 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:40.425943 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a49bfa2-f52c-40c4-9019-481b151d4aa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:40.426418 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a49bfa2-f52c-40c4-9019-481b151d4aa8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:40.427481 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a49bfa2-f52c-40c4-9019-481b151d4aa8 None admin] RBAC: Adding query filter params (role.id=6d5a3e6a22fa4f4fa47afcfce0246a1f, scope.project.id=7165d638ee164d27903602ef884fcf03, user.id=89ea290b3b2749e492283b8f12aa876e) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:25:40.428011 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a49bfa2-f52c-40c4-9019-481b151d4aa8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:40.429663 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a49bfa2-f52c-40c4-9019-481b151d4aa8 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:40.436090 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 96/208] 10.222.0.81 () {56 vars in 1374 bytes} [Wed Jul 20 22:25:40 2022] GET /identity/v3/role_assignments?user.id=89ea290b3b2749e492283b8f12aa876e&scope.project.id=7165d638ee164d27903602ef884fcf03&role.id=6d5a3e6a22fa4f4fa47afcfce0246a1f => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:25:41.179798 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71c01b8-6673-4421-b49a-f0f6195ff117 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:41.180170 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71c01b8-6673-4421-b49a-f0f6195ff117 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:41.180440 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71c01b8-6673-4421-b49a-f0f6195ff117 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:41.182366 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 113/209] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:25:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:25:41.189054 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed6f382-268a-433c-a4be-42156b4639ca None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:41.189361 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed6f382-268a-433c-a4be-42156b4639ca None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:41.189636 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed6f382-268a-433c-a4be-42156b4639ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:41.212179 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-1ed6f382-268a-433c-a4be-42156b4639ca None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:25:41.213635 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-1ed6f382-268a-433c-a4be-42156b4639ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:25:41.248341 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:41.256095 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078f8826-5f5e-48df-a30d-4000baf4ff75 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:41.263447 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078f8826-5f5e-48df-a30d-4000baf4ff75 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:41.263447 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078f8826-5f5e-48df-a30d-4000baf4ff75 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:41.263447 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 114/210] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:25:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:25:41.264466 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52577e7-1361-4f78-9ead-32f9e68102c3 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:41.264900 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52577e7-1361-4f78-9ead-32f9e68102c3 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:41.265412 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52577e7-1361-4f78-9ead-32f9e68102c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:41.282568 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-a52577e7-1361-4f78-9ead-32f9e68102c3 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:25:41.284170 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-a52577e7-1361-4f78-9ead-32f9e68102c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:25:41.288064 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 97/211] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:25:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 22:25:41.314661 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed6f382-268a-433c-a4be-42156b4639ca None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:41.356670 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 115/212] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:25:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 22:25:41.382138 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52577e7-1361-4f78-9ead-32f9e68102c3 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:41.434970 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed6f382-268a-433c-a4be-42156b4639ca None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:41.472044 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48953acf-4ef0-4291-9eaa-dfd953a52626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:41.473076 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48953acf-4ef0-4291-9eaa-dfd953a52626 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:41.473278 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48953acf-4ef0-4291-9eaa-dfd953a52626 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:41.473567 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48953acf-4ef0-4291-9eaa-dfd953a52626 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:41.485401 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48953acf-4ef0-4291-9eaa-dfd953a52626 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 22:25:41.489526 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52577e7-1361-4f78-9ead-32f9e68102c3 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:41.494757 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-48953acf-4ef0-4291-9eaa-dfd953a52626 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 22:25:41.494757 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:25:41.494757 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:25:41.494757 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:25:41.494757 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:25:41.494757 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:25:41.494757 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:25:41.494757 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:25:41.494757 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:25:41.494757 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:25:41.494757 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:25:41.494757 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:25:41.494757 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 22:25:41.494757 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 22:25:41.494757 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 22:25:41.494757 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 22:25:41.495677 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 22:25:41.495677 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 22:25:41.495677 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 22:25:41.495677 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 22:25:41.495677 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:25:41.495677 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:25:41.495677 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:25:41.495677 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:25:41.495677 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:25:41.495677 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:25:41.495677 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:25:41.495677 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:25:41.495677 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:25:41.495677 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:25:41.495677 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:25:41.495677 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:25:41.496461 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:25:41.496461 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:25:41.496461 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:25:41.496461 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:25:41.496461 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 22:25:41.496461 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 22:25:41.496461 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 22:25:41.496461 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 22:25:41.496461 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 22:25:41.496461 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 22:25:41.496461 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 22:25:41.496461 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:25:41.501318 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 98/213] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Jul 20 22:25:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 22:25:41.501860 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48953acf-4ef0-4291-9eaa-dfd953a52626 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:41.508798 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:41.529952 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf663f4-e211-4fa0-a3fd-555ddc45217f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:41.530933 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf663f4-e211-4fa0-a3fd-555ddc45217f None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:41.531344 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf663f4-e211-4fa0-a3fd-555ddc45217f None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:41.531717 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf663f4-e211-4fa0-a3fd-555ddc45217f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:41.533057 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf663f4-e211-4fa0-a3fd-555ddc45217f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:41.534651 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf663f4-e211-4fa0-a3fd-555ddc45217f None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:41.536582 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 116/214] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Jul 20 22:25:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 22:25:41.538441 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:41.542359 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf663f4-e211-4fa0-a3fd-555ddc45217f None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:41.619989 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48953acf-4ef0-4291-9eaa-dfd953a52626 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:41.651617 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf663f4-e211-4fa0-a3fd-555ddc45217f None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:41.657321 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb19887-c394-4c98-941b-47dcaf1db2f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:41.658198 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb19887-c394-4c98-941b-47dcaf1db2f2 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:41.658633 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb19887-c394-4c98-941b-47dcaf1db2f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:41.659115 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb19887-c394-4c98-941b-47dcaf1db2f2 None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:41.660124 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb19887-c394-4c98-941b-47dcaf1db2f2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:25:41.660649 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb19887-c394-4c98-941b-47dcaf1db2f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:41.662397 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb19887-c394-4c98-941b-47dcaf1db2f2 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:41.673458 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 99/215] 10.222.0.81 () {56 vars in 1101 bytes} [Wed Jul 20 22:25:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:25:41.679323 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb19887-c394-4c98-941b-47dcaf1db2f2 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:41.691472 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3b9d69-440d-4493-ac41-a303b72c25f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:41.692235 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3b9d69-440d-4493-ac41-a303b72c25f2 None admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:41.692621 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3b9d69-440d-4493-ac41-a303b72c25f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:41.693004 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3b9d69-440d-4493-ac41-a303b72c25f2 None admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:41.693892 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3b9d69-440d-4493-ac41-a303b72c25f2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:41.695508 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3b9d69-440d-4493-ac41-a303b72c25f2 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:41.779150 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:41.795037 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 117/216] 10.222.0.81 () {60 vars in 1132 bytes} [Wed Jul 20 22:25:41 2022] POST /identity/v3/users => generated 297 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:25:41.801100 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb19887-c394-4c98-941b-47dcaf1db2f2 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:41.838578 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9935d85-cd77-4e2e-8745-f67f604a4925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:41.839536 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9935d85-cd77-4e2e-8745-f67f604a4925 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:41.839825 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9935d85-cd77-4e2e-8745-f67f604a4925 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:41.840105 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9935d85-cd77-4e2e-8745-f67f604a4925 None admin] PATH_INFO: `/v3/users/56686220e81f4756a9ea2d7d411d7e32` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:41.856747 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9935d85-cd77-4e2e-8745-f67f604a4925 None admin] RBAC: Authorizing `identity:get_user(user_id=56686220e81f4756a9ea2d7d411d7e32)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:41.858384 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9935d85-cd77-4e2e-8745-f67f604a4925 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:41.860290 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 100/217] 10.222.0.81 () {56 vars in 1177 bytes} [Wed Jul 20 22:25:41 2022] GET /identity/v3/users/56686220e81f4756a9ea2d7d411d7e32 => generated 297 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 22:25:41.866249 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3b9d69-440d-4493-ac41-a303b72c25f2 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:41.881529 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88b29f1-3fa6-4568-bdbb-0b17b57f3a0b None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:41.881886 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88b29f1-3fa6-4568-bdbb-0b17b57f3a0b None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:41.882118 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88b29f1-3fa6-4568-bdbb-0b17b57f3a0b None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:41.883642 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 101/218] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:25:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 22:25:41.889380 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72ecfa1-524e-48bc-ad51-5441d1e158eb None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:41.889666 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72ecfa1-524e-48bc-ad51-5441d1e158eb None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:41.889952 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72ecfa1-524e-48bc-ad51-5441d1e158eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:41.908688 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-b72ecfa1-524e-48bc-ad51-5441d1e158eb None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:25:41.910031 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-b72ecfa1-524e-48bc-ad51-5441d1e158eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:25:41.917411 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:41.993384 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3b9d69-440d-4493-ac41-a303b72c25f2 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:41.995248 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 102/219] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:25:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 22:25:42.011936 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72ecfa1-524e-48bc-ad51-5441d1e158eb None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:42.031754 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e340dd46-4009-4993-90d7-6cf83508d042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:42.032452 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e340dd46-4009-4993-90d7-6cf83508d042 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:42.032746 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e340dd46-4009-4993-90d7-6cf83508d042 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:42.033033 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e340dd46-4009-4993-90d7-6cf83508d042 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:42.108694 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:42.110231 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e340dd46-4009-4993-90d7-6cf83508d042 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:42.111903 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e340dd46-4009-4993-90d7-6cf83508d042 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:42.128266 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72ecfa1-524e-48bc-ad51-5441d1e158eb None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:42.167592 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:42.172131 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ae80fa-fd73-4e8c-8a7f-09a043943cc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:42.173002 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ae80fa-fd73-4e8c-8a7f-09a043943cc5 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:42.173745 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ae80fa-fd73-4e8c-8a7f-09a043943cc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:42.173745 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ae80fa-fd73-4e8c-8a7f-09a043943cc5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:42.191446 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ae80fa-fd73-4e8c-8a7f-09a043943cc5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 22:25:42.200474 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-b4ae80fa-fd73-4e8c-8a7f-09a043943cc5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 22:25:42.200474 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:25:42.200474 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:25:42.200474 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:25:42.200474 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:25:42.200474 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:25:42.200474 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:25:42.200474 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:25:42.200474 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:25:42.200474 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:25:42.200474 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:25:42.200474 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:25:42.200474 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 22:25:42.200474 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 22:25:42.200474 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 22:25:42.200474 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 22:25:42.201393 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 22:25:42.201393 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 22:25:42.201393 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 22:25:42.201393 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 22:25:42.201393 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:25:42.201393 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:25:42.201393 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:25:42.201393 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:25:42.201393 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:25:42.201393 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:25:42.201393 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:25:42.201393 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:25:42.201393 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:25:42.201393 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:25:42.201393 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:25:42.201393 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:25:42.202212 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:25:42.202212 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:25:42.202212 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:25:42.202212 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:25:42.202212 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 22:25:42.202212 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 22:25:42.202212 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 22:25:42.202212 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 22:25:42.202212 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 22:25:42.202212 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 22:25:42.202212 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 22:25:42.202212 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:25:42.202212 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 103/220] 10.222.0.81 () {56 vars in 1096 bytes} [Wed Jul 20 22:25:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:25:42.207571 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ae80fa-fd73-4e8c-8a7f-09a043943cc5 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:42.254128 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 118/221] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Jul 20 22:25:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 22:25:42.261683 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e340dd46-4009-4993-90d7-6cf83508d042 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:42.336699 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ae80fa-fd73-4e8c-8a7f-09a043943cc5 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:42.376851 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e340dd46-4009-4993-90d7-6cf83508d042 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:42.385569 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17f2bae-203a-4d1f-b343-ef7673eb82fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:42.386314 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17f2bae-203a-4d1f-b343-ef7673eb82fc None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:42.386611 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17f2bae-203a-4d1f-b343-ef7673eb82fc None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:42.386895 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17f2bae-203a-4d1f-b343-ef7673eb82fc None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:42.388329 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17f2bae-203a-4d1f-b343-ef7673eb82fc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:25:42.388648 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17f2bae-203a-4d1f-b343-ef7673eb82fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:42.390549 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17f2bae-203a-4d1f-b343-ef7673eb82fc None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:42.402625 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 104/222] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Jul 20 22:25:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 22:25:42.412398 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17f2bae-203a-4d1f-b343-ef7673eb82fc None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:42.424201 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca70658f-7cf5-4ddd-bd3c-455c20dd4c91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:42.424744 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:42.426591 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca70658f-7cf5-4ddd-bd3c-455c20dd4c91 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:42.426885 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca70658f-7cf5-4ddd-bd3c-455c20dd4c91 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:42.427224 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca70658f-7cf5-4ddd-bd3c-455c20dd4c91 None admin] PATH_INFO: `/v3/projects/7165d638ee164d27903602ef884fcf03` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:42.428507 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca70658f-7cf5-4ddd-bd3c-455c20dd4c91 None admin] RBAC: Authorizing `identity:get_project(project_id=7165d638ee164d27903602ef884fcf03)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:42.429983 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca70658f-7cf5-4ddd-bd3c-455c20dd4c91 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:42.431918 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 119/223] 10.222.0.81 () {56 vars in 1186 bytes} [Wed Jul 20 22:25:42 2022] GET /identity/v3/projects/7165d638ee164d27903602ef884fcf03 => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 22:25:42.439999 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca70658f-7cf5-4ddd-bd3c-455c20dd4c91 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:42.456249 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:42.540227 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17f2bae-203a-4d1f-b343-ef7673eb82fc None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:42.583333 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3fe5e7-71ec-4142-97c3-bedf62d49391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:42.584721 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3fe5e7-71ec-4142-97c3-bedf62d49391 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:42.585231 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3fe5e7-71ec-4142-97c3-bedf62d49391 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:42.585703 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3fe5e7-71ec-4142-97c3-bedf62d49391 None admin] PATH_INFO: `/v3/users/89ea290b3b2749e492283b8f12aa876e` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:42.587183 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3fe5e7-71ec-4142-97c3-bedf62d49391 None admin] RBAC: Authorizing `identity:get_user(user_id=89ea290b3b2749e492283b8f12aa876e)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:42.589465 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3fe5e7-71ec-4142-97c3-bedf62d49391 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:42.591484 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 105/224] 10.222.0.81 () {56 vars in 1177 bytes} [Wed Jul 20 22:25:42 2022] GET /identity/v3/users/89ea290b3b2749e492283b8f12aa876e => generated 283 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:25:42.596975 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3fe5e7-71ec-4142-97c3-bedf62d49391 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:42.681708 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca70658f-7cf5-4ddd-bd3c-455c20dd4c91 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:42.720137 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:42.724466 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3470e17e-e496-4df0-a5a4-f8b62104d5fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:42.725389 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3470e17e-e496-4df0-a5a4-f8b62104d5fb None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:42.725870 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3470e17e-e496-4df0-a5a4-f8b62104d5fb None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:42.726342 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3470e17e-e496-4df0-a5a4-f8b62104d5fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:42.727442 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3470e17e-e496-4df0-a5a4-f8b62104d5fb None admin] RBAC: Adding query filter params (role.id=559942c4de1744169f8b5e0404ecde6b, scope.project.id=7165d638ee164d27903602ef884fcf03, user.id=56686220e81f4756a9ea2d7d411d7e32) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:25:42.727990 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3470e17e-e496-4df0-a5a4-f8b62104d5fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:42.729749 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3470e17e-e496-4df0-a5a4-f8b62104d5fb None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:42.735891 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 120/225] 10.222.0.81 () {56 vars in 1374 bytes} [Wed Jul 20 22:25:42 2022] GET /identity/v3/role_assignments?user.id=56686220e81f4756a9ea2d7d411d7e32&scope.project.id=7165d638ee164d27903602ef884fcf03&role.id=559942c4de1744169f8b5e0404ecde6b => generated 259 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:25:42.737337 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fcb9e0-6e95-41a4-bd7f-c9e1769c7bbf None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:42.737979 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fcb9e0-6e95-41a4-bd7f-c9e1769c7bbf None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:42.738468 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fcb9e0-6e95-41a4-bd7f-c9e1769c7bbf None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:42.740027 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3fe5e7-71ec-4142-97c3-bedf62d49391 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:42.740453 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 121/226] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:25:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:25:42.746821 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d15efb-2c2b-47ce-9c61-e560c6da6dbb None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:42.747173 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d15efb-2c2b-47ce-9c61-e560c6da6dbb None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:42.747421 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d15efb-2c2b-47ce-9c61-e560c6da6dbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:42.766758 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-98d15efb-2c2b-47ce-9c61-e560c6da6dbb None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:25:42.768012 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-98d15efb-2c2b-47ce-9c61-e560c6da6dbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:25:42.844881 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:42.845237 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5befd5-1571-4367-89f5-64ac1e8ef5a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:42.846129 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5befd5-1571-4367-89f5-64ac1e8ef5a0 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:42.846428 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5befd5-1571-4367-89f5-64ac1e8ef5a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:42.846709 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5befd5-1571-4367-89f5-64ac1e8ef5a0 None admin] PATH_INFO: `/v3/projects/7165d638ee164d27903602ef884fcf03` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:42.850239 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5befd5-1571-4367-89f5-64ac1e8ef5a0 None admin] RBAC: Authorizing `identity:get_project(project_id=7165d638ee164d27903602ef884fcf03)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:42.851763 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5befd5-1571-4367-89f5-64ac1e8ef5a0 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:42.853899 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 106/227] 10.222.0.81 () {56 vars in 1186 bytes} [Wed Jul 20 22:25:42 2022] GET /identity/v3/projects/7165d638ee164d27903602ef884fcf03 => generated 302 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:25:42.856628 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 122/228] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:25:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 22:25:42.864386 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5befd5-1571-4367-89f5-64ac1e8ef5a0 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:42.897670 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d15efb-2c2b-47ce-9c61-e560c6da6dbb None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:43.017735 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5befd5-1571-4367-89f5-64ac1e8ef5a0 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:43.066129 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d15efb-2c2b-47ce-9c61-e560c6da6dbb None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:43.076302 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3adab0-51a5-439d-9ef6-e9713cf7eef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:43.077232 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3adab0-51a5-439d-9ef6-e9713cf7eef8 None admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:43.077521 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3adab0-51a5-439d-9ef6-e9713cf7eef8 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:43.077802 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3adab0-51a5-439d-9ef6-e9713cf7eef8 None admin] PATH_INFO: `/v3/projects/7165d638ee164d27903602ef884fcf03/users/89ea290b3b2749e492283b8f12aa876e/roles/d89b6c92d35a401a8517f1d47f6be843` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:43.084352 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3adab0-51a5-439d-9ef6-e9713cf7eef8 None admin] RBAC: Authorizing `identity:create_grant(project_id=7165d638ee164d27903602ef884fcf03, user_id=89ea290b3b2749e492283b8f12aa876e, role_id=d89b6c92d35a401a8517f1d47f6be843)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:43.085826 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3adab0-51a5-439d-9ef6-e9713cf7eef8 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:43.103955 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750058f6-b68d-4d67-bd2a-1020f6c641b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:43.103955 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750058f6-b68d-4d67-bd2a-1020f6c641b4 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:43.103955 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750058f6-b68d-4d67-bd2a-1020f6c641b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:43.103955 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750058f6-b68d-4d67-bd2a-1020f6c641b4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:43.114810 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750058f6-b68d-4d67-bd2a-1020f6c641b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 22:25:43.124408 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-750058f6-b68d-4d67-bd2a-1020f6c641b4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 22:25:43.124408 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:25:43.124408 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:25:43.124408 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:25:43.124408 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:25:43.124408 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:25:43.124408 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:25:43.124408 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:25:43.124408 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:25:43.124408 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:25:43.124408 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:25:43.124408 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:25:43.124408 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 22:25:43.124408 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 22:25:43.124408 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 22:25:43.124408 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 22:25:43.125276 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 22:25:43.125276 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 22:25:43.125276 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 22:25:43.125276 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 22:25:43.125276 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:25:43.125276 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:25:43.125276 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:25:43.125276 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:25:43.125276 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:25:43.125276 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:25:43.125276 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:25:43.125276 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:25:43.125276 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:25:43.125276 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:25:43.125276 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:25:43.125276 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:25:43.126060 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:25:43.126060 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:25:43.126060 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:25:43.126060 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:25:43.126060 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 22:25:43.126060 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 22:25:43.126060 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 22:25:43.126060 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 22:25:43.126060 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 22:25:43.126060 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 22:25:43.126060 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 22:25:43.126060 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:25:43.126060 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 123/229] 10.222.0.81 () {56 vars in 1114 bytes} [Wed Jul 20 22:25:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 22:25:43.131922 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750058f6-b68d-4d67-bd2a-1020f6c641b4 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:43.168382 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 107/230] 10.222.0.81 () {58 vars in 1439 bytes} [Wed Jul 20 22:25:42 2022] PUT /identity/v3/projects/7165d638ee164d27903602ef884fcf03/users/89ea290b3b2749e492283b8f12aa876e/roles/d89b6c92d35a401a8517f1d47f6be843 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:25:43.168694 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:43.199309 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:43.269015 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750058f6-b68d-4d67-bd2a-1020f6c641b4 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:43.320761 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d89e534-eeed-46d9-bed9-225ba14f26a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:43.321448 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d89e534-eeed-46d9-bed9-225ba14f26a5 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:43.322388 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d89e534-eeed-46d9-bed9-225ba14f26a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:43.322673 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d89e534-eeed-46d9-bed9-225ba14f26a5 None admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:43.323469 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d89e534-eeed-46d9-bed9-225ba14f26a5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:25:43.323812 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d89e534-eeed-46d9-bed9-225ba14f26a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:43.325234 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d89e534-eeed-46d9-bed9-225ba14f26a5 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:43.339392 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 124/231] 10.222.0.81 () {56 vars in 1111 bytes} [Wed Jul 20 22:25:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 22:25:43.345239 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3adab0-51a5-439d-9ef6-e9713cf7eef8 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:43.460035 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3adab0-51a5-439d-9ef6-e9713cf7eef8 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:43.500683 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca855b43-1cdd-4152-a4ab-ae85a7091f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:43.501608 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca855b43-1cdd-4152-a4ab-ae85a7091f7b None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:43.502071 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca855b43-1cdd-4152-a4ab-ae85a7091f7b None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:43.502356 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca855b43-1cdd-4152-a4ab-ae85a7091f7b None admin] PATH_INFO: `/v3/users/89ea290b3b2749e492283b8f12aa876e` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:43.503623 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca855b43-1cdd-4152-a4ab-ae85a7091f7b None admin] RBAC: Authorizing `identity:get_user(user_id=89ea290b3b2749e492283b8f12aa876e)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:43.505085 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca855b43-1cdd-4152-a4ab-ae85a7091f7b None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:43.506733 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 108/232] 10.222.0.81 () {56 vars in 1177 bytes} [Wed Jul 20 22:25:43 2022] GET /identity/v3/users/89ea290b3b2749e492283b8f12aa876e => generated 283 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:25:43.512913 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d89e534-eeed-46d9-bed9-225ba14f26a5 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:43.647151 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d89e534-eeed-46d9-bed9-225ba14f26a5 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:43.679045 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:43.695182 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa168b8-cb72-43db-85ea-baaba0903176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:43.696074 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa168b8-cb72-43db-85ea-baaba0903176 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:43.696365 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa168b8-cb72-43db-85ea-baaba0903176 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:43.697167 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa168b8-cb72-43db-85ea-baaba0903176 None admin] PATH_INFO: `/v3/projects/7165d638ee164d27903602ef884fcf03` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:43.703777 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa168b8-cb72-43db-85ea-baaba0903176 None admin] RBAC: Authorizing `identity:get_project(project_id=7165d638ee164d27903602ef884fcf03)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:43.705249 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa168b8-cb72-43db-85ea-baaba0903176 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:43.706991 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 125/233] 10.222.0.81 () {56 vars in 1186 bytes} [Wed Jul 20 22:25:43 2022] GET /identity/v3/projects/7165d638ee164d27903602ef884fcf03 => generated 302 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:25:43.713491 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca855b43-1cdd-4152-a4ab-ae85a7091f7b None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:43.783405 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:43.828648 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca855b43-1cdd-4152-a4ab-ae85a7091f7b None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:43.873642 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccea2fb-63ba-48de-b481-8cd79562554f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:43.874547 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccea2fb-63ba-48de-b481-8cd79562554f None admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:43.874828 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccea2fb-63ba-48de-b481-8cd79562554f None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:43.875164 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccea2fb-63ba-48de-b481-8cd79562554f None admin] PATH_INFO: `/v3/projects/7165d638ee164d27903602ef884fcf03/users/89ea290b3b2749e492283b8f12aa876e/roles/6d5a3e6a22fa4f4fa47afcfce0246a1f` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:43.877382 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccea2fb-63ba-48de-b481-8cd79562554f None admin] RBAC: Authorizing `identity:create_grant(project_id=7165d638ee164d27903602ef884fcf03, user_id=89ea290b3b2749e492283b8f12aa876e, role_id=6d5a3e6a22fa4f4fa47afcfce0246a1f)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:43.878888 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccea2fb-63ba-48de-b481-8cd79562554f None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:44.022055 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 109/234] 10.222.0.81 () {58 vars in 1439 bytes} [Wed Jul 20 22:25:43 2022] PUT /identity/v3/projects/7165d638ee164d27903602ef884fcf03/users/89ea290b3b2749e492283b8f12aa876e/roles/6d5a3e6a22fa4f4fa47afcfce0246a1f => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:25:44.495720 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0488a3-3940-4d32-b1d5-af6ba53231d0 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:44.496225 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0488a3-3940-4d32-b1d5-af6ba53231d0 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:44.496325 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0488a3-3940-4d32-b1d5-af6ba53231d0 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:44.497806 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 126/235] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:25:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:25:44.502881 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372ab234-2843-458a-a27b-13f7f42b1eac None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:44.503276 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372ab234-2843-458a-a27b-13f7f42b1eac None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:44.503482 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372ab234-2843-458a-a27b-13f7f42b1eac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:44.527999 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-372ab234-2843-458a-a27b-13f7f42b1eac None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:25:44.529227 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-372ab234-2843-458a-a27b-13f7f42b1eac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:25:44.596329 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77e3b9c-8324-4a24-a4f3-48daf6ec0146 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:44.596636 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77e3b9c-8324-4a24-a4f3-48daf6ec0146 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:44.596895 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77e3b9c-8324-4a24-a4f3-48daf6ec0146 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:44.599678 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 127/236] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:25:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:25:44.604321 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edb1ce8-fbea-4f27-9104-45b74ed317e0 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:44.604645 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edb1ce8-fbea-4f27-9104-45b74ed317e0 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:44.604927 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edb1ce8-fbea-4f27-9104-45b74ed317e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:44.633317 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 110/237] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:25:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 22:25:44.633524 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-4edb1ce8-fbea-4f27-9104-45b74ed317e0 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:25:44.634975 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-4edb1ce8-fbea-4f27-9104-45b74ed317e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:25:44.656203 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372ab234-2843-458a-a27b-13f7f42b1eac None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:44.723869 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:44.768979 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372ab234-2843-458a-a27b-13f7f42b1eac None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:44.790345 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 128/238] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:25:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 22:25:44.807325 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9eeaea-7014-4d7b-ace8-c1cf3bbfec1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:44.808165 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9eeaea-7014-4d7b-ace8-c1cf3bbfec1e None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:44.808451 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9eeaea-7014-4d7b-ace8-c1cf3bbfec1e None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:44.808731 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9eeaea-7014-4d7b-ace8-c1cf3bbfec1e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:44.817203 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edb1ce8-fbea-4f27-9104-45b74ed317e0 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:44.820284 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9eeaea-7014-4d7b-ace8-c1cf3bbfec1e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 22:25:44.830065 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-fc9eeaea-7014-4d7b-ace8-c1cf3bbfec1e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 22:25:44.830065 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:25:44.830065 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:25:44.830065 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:25:44.830065 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:25:44.830065 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:25:44.830065 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:25:44.830065 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:25:44.830065 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:25:44.830065 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:25:44.830065 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:25:44.830065 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:25:44.830065 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 22:25:44.830065 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 22:25:44.830065 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 22:25:44.830065 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 22:25:44.831217 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 22:25:44.831217 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 22:25:44.831217 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 22:25:44.831217 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 22:25:44.831217 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:25:44.831217 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:25:44.831217 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:25:44.831217 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:25:44.831217 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:25:44.831217 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:25:44.831217 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:25:44.831217 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:25:44.831217 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:25:44.831217 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:25:44.831217 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:25:44.831217 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:25:44.832187 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:25:44.832187 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:25:44.832187 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:25:44.832187 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:25:44.832187 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 22:25:44.832187 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 22:25:44.832187 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 22:25:44.832187 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 22:25:44.832187 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 22:25:44.832187 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 22:25:44.832187 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 22:25:44.832187 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:25:44.833592 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 111/239] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Jul 20 22:25:44 2022] GET /identity/v3/roles/reader => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:25:44.839820 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9eeaea-7014-4d7b-ace8-c1cf3bbfec1e None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:44.944453 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9eeaea-7014-4d7b-ace8-c1cf3bbfec1e None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:44.961686 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:44.979958 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adac0da6-8bdc-4bfb-b954-67e98187fca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:44.980653 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adac0da6-8bdc-4bfb-b954-67e98187fca0 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:44.980943 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adac0da6-8bdc-4bfb-b954-67e98187fca0 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:44.981230 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adac0da6-8bdc-4bfb-b954-67e98187fca0 None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:44.981991 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adac0da6-8bdc-4bfb-b954-67e98187fca0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:25:44.982306 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adac0da6-8bdc-4bfb-b954-67e98187fca0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:44.983739 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adac0da6-8bdc-4bfb-b954-67e98187fca0 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:44.990861 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edb1ce8-fbea-4f27-9104-45b74ed317e0 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:44.992875 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 112/240] 10.222.0.81 () {56 vars in 1101 bytes} [Wed Jul 20 22:25:44 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 22:25:44.998820 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adac0da6-8bdc-4bfb-b954-67e98187fca0 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:45.035804 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:45.085030 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf120c44-b4bb-492c-97d0-6c6bccdc68dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:45.085853 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf120c44-b4bb-492c-97d0-6c6bccdc68dd None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:45.086136 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf120c44-b4bb-492c-97d0-6c6bccdc68dd None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:45.086416 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf120c44-b4bb-492c-97d0-6c6bccdc68dd None admin] PATH_INFO: `/v3/domains/default` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:45.090410 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf120c44-b4bb-492c-97d0-6c6bccdc68dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:45.091918 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf120c44-b4bb-492c-97d0-6c6bccdc68dd None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:45.093883 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 129/241] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Jul 20 22:25:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 22:25:45.100360 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf120c44-b4bb-492c-97d0-6c6bccdc68dd None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:45.106104 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adac0da6-8bdc-4bfb-b954-67e98187fca0 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:45.142567 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a0ddfb-fc9a-4455-a181-91df5467c3a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:45.143553 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a0ddfb-fc9a-4455-a181-91df5467c3a3 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:45.143895 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a0ddfb-fc9a-4455-a181-91df5467c3a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:45.144193 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a0ddfb-fc9a-4455-a181-91df5467c3a3 None admin] PATH_INFO: `/v3/users/6e399a52b2b746e4978ef90c96c96594` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:45.158129 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a0ddfb-fc9a-4455-a181-91df5467c3a3 None admin] RBAC: Authorizing `identity:get_user(user_id=6e399a52b2b746e4978ef90c96c96594)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:45.159784 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a0ddfb-fc9a-4455-a181-91df5467c3a3 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:45.161500 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 113/242] 10.222.0.81 () {56 vars in 1177 bytes} [Wed Jul 20 22:25:44 2022] GET /identity/v3/users/6e399a52b2b746e4978ef90c96c96594 => generated 297 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 22:25:45.167529 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a0ddfb-fc9a-4455-a181-91df5467c3a3 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:45.208997 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:45.270403 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a0ddfb-fc9a-4455-a181-91df5467c3a3 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:45.318231 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf120c44-b4bb-492c-97d0-6c6bccdc68dd None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:45.339727 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355a1671-93aa-43f6-a734-0d4e3ad93dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:45.340527 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355a1671-93aa-43f6-a734-0d4e3ad93dd6 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:45.340824 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355a1671-93aa-43f6-a734-0d4e3ad93dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:45.341110 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355a1671-93aa-43f6-a734-0d4e3ad93dd6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:45.357873 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe93de2-dbbf-458e-8ac3-63dd3e46ab4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:45.359954 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe93de2-dbbf-458e-8ac3-63dd3e46ab4f None admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:45.360676 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe93de2-dbbf-458e-8ac3-63dd3e46ab4f None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:45.361823 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe93de2-dbbf-458e-8ac3-63dd3e46ab4f None admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:45.362914 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe93de2-dbbf-458e-8ac3-63dd3e46ab4f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:45.365809 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe93de2-dbbf-458e-8ac3-63dd3e46ab4f None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:45.441879 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355a1671-93aa-43f6-a734-0d4e3ad93dd6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:45.443487 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355a1671-93aa-43f6-a734-0d4e3ad93dd6 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:45.462035 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 130/243] 10.222.0.81 () {60 vars in 1132 bytes} [Wed Jul 20 22:25:45 2022] POST /identity/v3/users => generated 293 bytes in 362 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:25:45.465437 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87de9a09-1e00-4c45-8c00-40a3b01d0bea None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:45.465991 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87de9a09-1e00-4c45-8c00-40a3b01d0bea None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:45.466595 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87de9a09-1e00-4c45-8c00-40a3b01d0bea None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:45.468323 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 131/244] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:25:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:25:45.474802 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4706359-0486-43da-84a5-e35a40fc5b87 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:45.475476 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4706359-0486-43da-84a5-e35a40fc5b87 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:45.475965 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4706359-0486-43da-84a5-e35a40fc5b87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:45.489784 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:45.496915 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-c4706359-0486-43da-84a5-e35a40fc5b87 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:25:45.498955 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-c4706359-0486-43da-84a5-e35a40fc5b87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:25:45.548764 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:45.582037 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 132/245] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:25:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 22:25:45.608393 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4706359-0486-43da-84a5-e35a40fc5b87 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:45.646333 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 114/246] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Jul 20 22:25:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 479 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 22:25:45.651726 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355a1671-93aa-43f6-a734-0d4e3ad93dd6 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:45.727146 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4706359-0486-43da-84a5-e35a40fc5b87 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:45.766186 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355a1671-93aa-43f6-a734-0d4e3ad93dd6 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:45.770653 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd2f055-2392-41cd-8e28-0a3a590351b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:45.771810 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd2f055-2392-41cd-8e28-0a3a590351b2 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:45.772311 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd2f055-2392-41cd-8e28-0a3a590351b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:45.772810 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd2f055-2392-41cd-8e28-0a3a590351b2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:45.785035 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd2f055-2392-41cd-8e28-0a3a590351b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 22:25:45.794024 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-ffd2f055-2392-41cd-8e28-0a3a590351b2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 22:25:45.794024 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:25:45.794024 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:25:45.794024 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:25:45.794024 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:25:45.794024 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:25:45.794024 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:25:45.794024 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:25:45.794024 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:25:45.794024 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:25:45.794024 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:25:45.794024 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:25:45.794024 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 22:25:45.794024 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 22:25:45.794024 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 22:25:45.794024 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 22:25:45.795000 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 22:25:45.795000 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 22:25:45.795000 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 22:25:45.795000 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 22:25:45.795000 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:25:45.795000 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:25:45.795000 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:25:45.795000 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:25:45.795000 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:25:45.795000 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:25:45.795000 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:25:45.795000 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:25:45.795000 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:25:45.795000 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:25:45.795000 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:25:45.795000 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:25:45.795827 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:25:45.795827 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:25:45.795827 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:25:45.795827 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:25:45.795827 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 22:25:45.795827 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 22:25:45.795827 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 22:25:45.795827 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 22:25:45.795827 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 22:25:45.795827 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 22:25:45.795827 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 22:25:45.795827 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:25:45.798534 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 133/247] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Jul 20 22:25:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 22:25:45.804595 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd2f055-2392-41cd-8e28-0a3a590351b2 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:45.812634 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6abeef2-7eb8-4b46-8b05-76b268505e94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:45.813838 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6abeef2-7eb8-4b46-8b05-76b268505e94 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:45.814484 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6abeef2-7eb8-4b46-8b05-76b268505e94 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:45.814962 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6abeef2-7eb8-4b46-8b05-76b268505e94 None admin] PATH_INFO: `/v3/projects/7165d638ee164d27903602ef884fcf03` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:45.817291 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6abeef2-7eb8-4b46-8b05-76b268505e94 None admin] RBAC: Authorizing `identity:get_project(project_id=7165d638ee164d27903602ef884fcf03)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:45.819757 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6abeef2-7eb8-4b46-8b05-76b268505e94 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:45.822607 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 115/248] 10.222.0.81 () {56 vars in 1186 bytes} [Wed Jul 20 22:25:45 2022] GET /identity/v3/projects/7165d638ee164d27903602ef884fcf03 => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 22:25:45.829645 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6abeef2-7eb8-4b46-8b05-76b268505e94 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:45.834825 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:45.852848 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:45.937928 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd2f055-2392-41cd-8e28-0a3a590351b2 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:45.960494 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6abeef2-7eb8-4b46-8b05-76b268505e94 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:45.984234 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e505c2-0210-4c3d-8160-b973941aced6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:45.984923 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e505c2-0210-4c3d-8160-b973941aced6 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:45.985212 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e505c2-0210-4c3d-8160-b973941aced6 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:45.985496 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e505c2-0210-4c3d-8160-b973941aced6 None admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:45.986254 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e505c2-0210-4c3d-8160-b973941aced6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:25:45.986565 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e505c2-0210-4c3d-8160-b973941aced6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:45.988423 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e505c2-0210-4c3d-8160-b973941aced6 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:45.998529 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 134/249] 10.222.0.81 () {56 vars in 1101 bytes} [Wed Jul 20 22:25:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 22:25:45.999909 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bf187d-21dd-41aa-b613-9a088d50e326 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:46.000206 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bf187d-21dd-41aa-b613-9a088d50e326 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:46.000619 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bf187d-21dd-41aa-b613-9a088d50e326 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:46.007651 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 135/250] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:25:45 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:25:46.008249 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bf187d-21dd-41aa-b613-9a088d50e326 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:46.011735 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c0fc5f-d655-418d-b0c8-4b1fdeb203cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:46.012424 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c0fc5f-d655-418d-b0c8-4b1fdeb203cd None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:46.012710 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c0fc5f-d655-418d-b0c8-4b1fdeb203cd None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:46.012984 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c0fc5f-d655-418d-b0c8-4b1fdeb203cd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:46.013825 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c0fc5f-d655-418d-b0c8-4b1fdeb203cd None admin] RBAC: Adding query filter params (role.id=5b2ef0feadf94c059dffbd7d8bc03aa4, scope.project.id=7165d638ee164d27903602ef884fcf03, user.id=6e399a52b2b746e4978ef90c96c96594) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:25:46.014204 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c0fc5f-d655-418d-b0c8-4b1fdeb203cd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:46.015718 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c0fc5f-d655-418d-b0c8-4b1fdeb203cd None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:46.022419 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 116/251] 10.222.0.81 () {56 vars in 1374 bytes} [Wed Jul 20 22:25:45 2022] GET /identity/v3/role_assignments?user.id=6e399a52b2b746e4978ef90c96c96594&scope.project.id=7165d638ee164d27903602ef884fcf03&role.id=5b2ef0feadf94c059dffbd7d8bc03aa4 => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:25:46.025383 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9eee4c-b168-4198-8f07-20112825e404 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:46.025677 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9eee4c-b168-4198-8f07-20112825e404 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:46.025957 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9eee4c-b168-4198-8f07-20112825e404 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:46.045011 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-1f9eee4c-b168-4198-8f07-20112825e404 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:25:46.046331 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-1f9eee4c-b168-4198-8f07-20112825e404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:25:46.119086 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:46.123271 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:46.126482 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bf187d-21dd-41aa-b613-9a088d50e326 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:46.134735 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 117/252] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:25:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 22:25:46.163665 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9eee4c-b168-4198-8f07-20112825e404 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:46.203348 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef14dda2-0a39-48a3-a003-f11e4a2ad4c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:46.204371 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef14dda2-0a39-48a3-a003-f11e4a2ad4c7 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:46.204659 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef14dda2-0a39-48a3-a003-f11e4a2ad4c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:46.204942 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef14dda2-0a39-48a3-a003-f11e4a2ad4c7 None admin] PATH_INFO: `/v3/users/56686220e81f4756a9ea2d7d411d7e32` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:46.211440 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef14dda2-0a39-48a3-a003-f11e4a2ad4c7 None admin] RBAC: Authorizing `identity:get_user(user_id=56686220e81f4756a9ea2d7d411d7e32)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:46.213525 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef14dda2-0a39-48a3-a003-f11e4a2ad4c7 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:46.217943 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 136/253] 10.222.0.81 () {56 vars in 1177 bytes} [Wed Jul 20 22:25:46 2022] GET /identity/v3/users/56686220e81f4756a9ea2d7d411d7e32 => generated 297 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:25:46.222357 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef14dda2-0a39-48a3-a003-f11e4a2ad4c7 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:46.327565 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9eee4c-b168-4198-8f07-20112825e404 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:46.354229 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef14dda2-0a39-48a3-a003-f11e4a2ad4c7 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:46.386984 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deae6bca-5c43-42b3-acd7-763fe8937b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:46.387709 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deae6bca-5c43-42b3-acd7-763fe8937b8a None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:46.388043 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deae6bca-5c43-42b3-acd7-763fe8937b8a None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:46.388327 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deae6bca-5c43-42b3-acd7-763fe8937b8a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:46.390464 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165c684f-7f31-40b3-b2c1-6fb6bee617e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:46.391374 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165c684f-7f31-40b3-b2c1-6fb6bee617e9 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:46.391663 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165c684f-7f31-40b3-b2c1-6fb6bee617e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:46.391946 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165c684f-7f31-40b3-b2c1-6fb6bee617e9 None admin] PATH_INFO: `/v3/projects/7165d638ee164d27903602ef884fcf03` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:46.393162 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165c684f-7f31-40b3-b2c1-6fb6bee617e9 None admin] RBAC: Authorizing `identity:get_project(project_id=7165d638ee164d27903602ef884fcf03)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:46.394612 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165c684f-7f31-40b3-b2c1-6fb6bee617e9 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:46.396579 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 137/254] 10.222.0.81 () {56 vars in 1186 bytes} [Wed Jul 20 22:25:46 2022] GET /identity/v3/projects/7165d638ee164d27903602ef884fcf03 => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:25:46.402039 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165c684f-7f31-40b3-b2c1-6fb6bee617e9 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:46.413373 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deae6bca-5c43-42b3-acd7-763fe8937b8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 22:25:46.422311 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-deae6bca-5c43-42b3-acd7-763fe8937b8a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 22:25:46.422311 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:25:46.422311 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:25:46.422311 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:25:46.422311 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:25:46.422311 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:25:46.422311 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:25:46.422311 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:25:46.422311 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:25:46.422311 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:25:46.422311 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:25:46.422311 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:25:46.422311 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 22:25:46.422311 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 22:25:46.422311 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 22:25:46.422311 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 22:25:46.423252 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 22:25:46.423252 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 22:25:46.423252 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 22:25:46.423252 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 22:25:46.423252 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:25:46.423252 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:25:46.423252 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:25:46.423252 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:25:46.423252 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:25:46.423252 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:25:46.423252 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:25:46.423252 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:25:46.423252 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:25:46.423252 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:25:46.423252 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:25:46.423252 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:25:46.424155 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:25:46.424155 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:25:46.424155 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:25:46.424155 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:25:46.424155 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 22:25:46.424155 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 22:25:46.424155 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 22:25:46.424155 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 22:25:46.424155 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 22:25:46.424155 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 22:25:46.424155 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 22:25:46.424155 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:25:46.426851 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 118/255] 10.222.0.81 () {56 vars in 1096 bytes} [Wed Jul 20 22:25:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 22:25:46.437854 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:46.439110 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deae6bca-5c43-42b3-acd7-763fe8937b8a None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:46.513294 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165c684f-7f31-40b3-b2c1-6fb6bee617e9 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:46.563210 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e771ee-27ad-45bb-aaf8-26d7c802ee28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:46.564122 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e771ee-27ad-45bb-aaf8-26d7c802ee28 None admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:46.564410 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e771ee-27ad-45bb-aaf8-26d7c802ee28 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:46.564690 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e771ee-27ad-45bb-aaf8-26d7c802ee28 None admin] PATH_INFO: `/v3/projects/7165d638ee164d27903602ef884fcf03/users/56686220e81f4756a9ea2d7d411d7e32/roles/559942c4de1744169f8b5e0404ecde6b` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:46.567910 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e771ee-27ad-45bb-aaf8-26d7c802ee28 None admin] RBAC: Authorizing `identity:create_grant(project_id=7165d638ee164d27903602ef884fcf03, user_id=56686220e81f4756a9ea2d7d411d7e32, role_id=559942c4de1744169f8b5e0404ecde6b)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:46.568408 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:46.569395 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e771ee-27ad-45bb-aaf8-26d7c802ee28 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:46.615177 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deae6bca-5c43-42b3-acd7-763fe8937b8a None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:46.656106 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 138/256] 10.222.0.81 () {58 vars in 1439 bytes} [Wed Jul 20 22:25:46 2022] PUT /identity/v3/projects/7165d638ee164d27903602ef884fcf03/users/56686220e81f4756a9ea2d7d411d7e32/roles/559942c4de1744169f8b5e0404ecde6b => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 22:25:46.661840 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e37d966-33bf-4165-90fc-928941d5e14e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:46.662544 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e37d966-33bf-4165-90fc-928941d5e14e None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:46.662835 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e37d966-33bf-4165-90fc-928941d5e14e None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:46.670237 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e37d966-33bf-4165-90fc-928941d5e14e None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:46.670237 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e37d966-33bf-4165-90fc-928941d5e14e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:25:46.670237 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e37d966-33bf-4165-90fc-928941d5e14e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:46.670237 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e37d966-33bf-4165-90fc-928941d5e14e None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:46.677619 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 119/257] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Jul 20 22:25:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:25:46.683146 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e771ee-27ad-45bb-aaf8-26d7c802ee28 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:46.818096 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:46.827761 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e771ee-27ad-45bb-aaf8-26d7c802ee28 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:46.917756 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb96265b-6a66-4beb-bb8e-c00f0ceb7e50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:46.918829 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb96265b-6a66-4beb-bb8e-c00f0ceb7e50 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:46.919149 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb96265b-6a66-4beb-bb8e-c00f0ceb7e50 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:46.919431 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb96265b-6a66-4beb-bb8e-c00f0ceb7e50 None admin] PATH_INFO: `/v3/users/89ea290b3b2749e492283b8f12aa876e` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:46.920680 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb96265b-6a66-4beb-bb8e-c00f0ceb7e50 None admin] RBAC: Authorizing `identity:get_user(user_id=89ea290b3b2749e492283b8f12aa876e)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:46.922151 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb96265b-6a66-4beb-bb8e-c00f0ceb7e50 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:46.924211 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 139/258] 10.222.0.81 () {56 vars in 1177 bytes} [Wed Jul 20 22:25:46 2022] GET /identity/v3/users/89ea290b3b2749e492283b8f12aa876e => generated 283 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:25:46.929469 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e37d966-33bf-4165-90fc-928941d5e14e None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:46.949651 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b411c897-f57d-4344-8b9d-0aba89df0d61 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:46.949983 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b411c897-f57d-4344-8b9d-0aba89df0d61 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:46.950218 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b411c897-f57d-4344-8b9d-0aba89df0d61 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:46.951670 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 140/259] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:25:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 22:25:46.965586 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba50ea72-093a-4d89-965f-6904a9185d85 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:46.965919 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba50ea72-093a-4d89-965f-6904a9185d85 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:46.966156 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba50ea72-093a-4d89-965f-6904a9185d85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:46.988537 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-ba50ea72-093a-4d89-965f-6904a9185d85 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:25:46.989756 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-ba50ea72-093a-4d89-965f-6904a9185d85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:25:47.113556 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e37d966-33bf-4165-90fc-928941d5e14e None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:47.118763 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 141/260] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:25:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 22:25:47.142536 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:47.150880 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b04f4e0-d36b-4e36-b39f-3f7ab452eee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:47.151587 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b04f4e0-d36b-4e36-b39f-3f7ab452eee2 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:47.151878 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b04f4e0-d36b-4e36-b39f-3f7ab452eee2 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:47.152161 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b04f4e0-d36b-4e36-b39f-3f7ab452eee2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:47.157940 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba50ea72-093a-4d89-965f-6904a9185d85 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:47.288459 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b04f4e0-d36b-4e36-b39f-3f7ab452eee2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:47.289963 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b04f4e0-d36b-4e36-b39f-3f7ab452eee2 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:47.332815 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:47.349441 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba50ea72-093a-4d89-965f-6904a9185d85 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:47.424715 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63efc13f-cc5e-4b4d-9e8e-d22666c259fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:47.425578 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63efc13f-cc5e-4b4d-9e8e-d22666c259fa None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:47.425872 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63efc13f-cc5e-4b4d-9e8e-d22666c259fa None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:47.426157 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63efc13f-cc5e-4b4d-9e8e-d22666c259fa None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:47.448057 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63efc13f-cc5e-4b4d-9e8e-d22666c259fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 22:25:47.456815 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-63efc13f-cc5e-4b4d-9e8e-d22666c259fa None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 22:25:47.456815 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:25:47.456815 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:25:47.456815 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:25:47.456815 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:25:47.456815 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:25:47.456815 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:25:47.456815 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:25:47.456815 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:25:47.456815 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:25:47.456815 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:25:47.456815 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:25:47.456815 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 22:25:47.456815 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 22:25:47.456815 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 22:25:47.456815 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 22:25:47.457713 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 22:25:47.457713 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 22:25:47.457713 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 22:25:47.457713 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 22:25:47.457713 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:25:47.457713 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:25:47.457713 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:25:47.457713 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:25:47.457713 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:25:47.457713 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:25:47.457713 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:25:47.457713 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:25:47.457713 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:25:47.457713 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:25:47.457713 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:25:47.457713 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:25:47.458478 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:25:47.458478 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:25:47.458478 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:25:47.458478 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:25:47.458478 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 22:25:47.458478 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 22:25:47.458478 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 22:25:47.458478 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 22:25:47.458478 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 22:25:47.458478 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 22:25:47.458478 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 22:25:47.458478 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:25:47.458478 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 142/261] 10.222.0.81 () {56 vars in 1114 bytes} [Wed Jul 20 22:25:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 22:25:47.468489 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63efc13f-cc5e-4b4d-9e8e-d22666c259fa None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:47.479997 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 120/262] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Jul 20 22:25:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 551 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 22:25:47.485520 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b04f4e0-d36b-4e36-b39f-3f7ab452eee2 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:47.515397 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:47.590483 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b04f4e0-d36b-4e36-b39f-3f7ab452eee2 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:47.640637 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32b2fe8-a254-4114-a3cb-9c95a4d4c6ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:47.641527 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32b2fe8-a254-4114-a3cb-9c95a4d4c6ab None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:47.641815 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32b2fe8-a254-4114-a3cb-9c95a4d4c6ab None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:47.642091 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32b2fe8-a254-4114-a3cb-9c95a4d4c6ab None admin] PATH_INFO: `/v3/projects/7165d638ee164d27903602ef884fcf03` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:47.644313 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32b2fe8-a254-4114-a3cb-9c95a4d4c6ab None admin] RBAC: Authorizing `identity:get_project(project_id=7165d638ee164d27903602ef884fcf03)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:47.645758 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32b2fe8-a254-4114-a3cb-9c95a4d4c6ab None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:47.656369 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 121/263] 10.222.0.81 () {56 vars in 1186 bytes} [Wed Jul 20 22:25:47 2022] GET /identity/v3/projects/7165d638ee164d27903602ef884fcf03 => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 22:25:47.659180 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63efc13f-cc5e-4b4d-9e8e-d22666c259fa None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:47.664280 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32b2fe8-a254-4114-a3cb-9c95a4d4c6ab None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:47.742572 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52382546-ddbc-40f1-b41f-2a10dc97aa6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:47.743245 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52382546-ddbc-40f1-b41f-2a10dc97aa6b None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:47.744044 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52382546-ddbc-40f1-b41f-2a10dc97aa6b None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:47.744329 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52382546-ddbc-40f1-b41f-2a10dc97aa6b None admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:47.745087 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52382546-ddbc-40f1-b41f-2a10dc97aa6b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:25:47.745400 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52382546-ddbc-40f1-b41f-2a10dc97aa6b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:47.746790 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52382546-ddbc-40f1-b41f-2a10dc97aa6b None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:47.758643 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 143/264] 10.222.0.81 () {56 vars in 1111 bytes} [Wed Jul 20 22:25:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:25:47.764781 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52382546-ddbc-40f1-b41f-2a10dc97aa6b None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:47.784338 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:47.860233 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:47.880915 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32b2fe8-a254-4114-a3cb-9c95a4d4c6ab None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:47.954966 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9164d0-93a3-4e88-a5c5-480156e31f1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:47.955672 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9164d0-93a3-4e88-a5c5-480156e31f1f None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:47.955963 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9164d0-93a3-4e88-a5c5-480156e31f1f None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:47.956245 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9164d0-93a3-4e88-a5c5-480156e31f1f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:47.957079 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9164d0-93a3-4e88-a5c5-480156e31f1f None admin] RBAC: Adding query filter params (role.id=d89b6c92d35a401a8517f1d47f6be843, scope.project.id=7165d638ee164d27903602ef884fcf03, user.id=89ea290b3b2749e492283b8f12aa876e) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:25:47.957398 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9164d0-93a3-4e88-a5c5-480156e31f1f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:47.958846 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9164d0-93a3-4e88-a5c5-480156e31f1f None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:47.965779 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52382546-ddbc-40f1-b41f-2a10dc97aa6b None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:47.977089 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 122/265] 10.222.0.81 () {56 vars in 1374 bytes} [Wed Jul 20 22:25:47 2022] GET /identity/v3/role_assignments?user.id=89ea290b3b2749e492283b8f12aa876e&scope.project.id=7165d638ee164d27903602ef884fcf03&role.id=d89b6c92d35a401a8517f1d47f6be843 => generated 608 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 22:25:48.019337 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001243e6-0c04-4d4d-8580-1196063ff245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:48.020167 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001243e6-0c04-4d4d-8580-1196063ff245 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:48.020459 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001243e6-0c04-4d4d-8580-1196063ff245 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:48.020746 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001243e6-0c04-4d4d-8580-1196063ff245 None admin] PATH_INFO: `/v3/users/89ea290b3b2749e492283b8f12aa876e` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:48.022024 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001243e6-0c04-4d4d-8580-1196063ff245 None admin] RBAC: Authorizing `identity:get_user(user_id=89ea290b3b2749e492283b8f12aa876e)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:48.023584 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001243e6-0c04-4d4d-8580-1196063ff245 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:48.025650 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 144/266] 10.222.0.81 () {56 vars in 1177 bytes} [Wed Jul 20 22:25:47 2022] GET /identity/v3/users/89ea290b3b2749e492283b8f12aa876e => generated 283 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 22:25:48.031164 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9164d0-93a3-4e88-a5c5-480156e31f1f None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:48.159036 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9164d0-93a3-4e88-a5c5-480156e31f1f None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:48.196927 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8666ad4-9b24-45d5-9db6-a7707beb0b3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:48.197066 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8666ad4-9b24-45d5-9db6-a7707beb0b3b None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:48.197305 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8666ad4-9b24-45d5-9db6-a7707beb0b3b None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:48.197582 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8666ad4-9b24-45d5-9db6-a7707beb0b3b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:48.224995 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:48.277941 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8666ad4-9b24-45d5-9db6-a7707beb0b3b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:48.279352 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8666ad4-9b24-45d5-9db6-a7707beb0b3b None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:48.389180 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 123/267] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Jul 20 22:25:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 22:25:48.395135 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001243e6-0c04-4d4d-8580-1196063ff245 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:48.491797 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:48.514652 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001243e6-0c04-4d4d-8580-1196063ff245 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:48.551633 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f807409-a980-421c-9eaa-dd82f8af59ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:48.552490 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f807409-a980-421c-9eaa-dd82f8af59ea None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:48.552782 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f807409-a980-421c-9eaa-dd82f8af59ea None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:48.553062 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f807409-a980-421c-9eaa-dd82f8af59ea None admin] PATH_INFO: `/v3/projects/7165d638ee164d27903602ef884fcf03` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:48.554269 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f807409-a980-421c-9eaa-dd82f8af59ea None admin] RBAC: Authorizing `identity:get_project(project_id=7165d638ee164d27903602ef884fcf03)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:48.556354 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f807409-a980-421c-9eaa-dd82f8af59ea None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:48.558381 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 145/268] 10.222.0.81 () {56 vars in 1186 bytes} [Wed Jul 20 22:25:48 2022] GET /identity/v3/projects/7165d638ee164d27903602ef884fcf03 => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:25:48.570284 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8666ad4-9b24-45d5-9db6-a7707beb0b3b None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:48.631107 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:48.688524 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8666ad4-9b24-45d5-9db6-a7707beb0b3b None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:48.741145 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7e4a4b-a6cc-4abb-ae9a-e6fe9af2f89a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:48.741821 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7e4a4b-a6cc-4abb-ae9a-e6fe9af2f89a None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:48.742108 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7e4a4b-a6cc-4abb-ae9a-e6fe9af2f89a None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:48.742386 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7e4a4b-a6cc-4abb-ae9a-e6fe9af2f89a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:48.743761 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7e4a4b-a6cc-4abb-ae9a-e6fe9af2f89a None admin] RBAC: Adding query filter params (role.id=6d5a3e6a22fa4f4fa47afcfce0246a1f, scope.project.id=7165d638ee164d27903602ef884fcf03, user.id=89ea290b3b2749e492283b8f12aa876e) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:25:48.744077 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7e4a4b-a6cc-4abb-ae9a-e6fe9af2f89a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:48.745555 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7e4a4b-a6cc-4abb-ae9a-e6fe9af2f89a None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:48.751929 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 124/269] 10.222.0.81 () {56 vars in 1374 bytes} [Wed Jul 20 22:25:48 2022] GET /identity/v3/role_assignments?user.id=89ea290b3b2749e492283b8f12aa876e&scope.project.id=7165d638ee164d27903602ef884fcf03&role.id=6d5a3e6a22fa4f4fa47afcfce0246a1f => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:25:48.969047 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac205bf3-fab8-4da5-b292-461128758c3d None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:48.969552 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac205bf3-fab8-4da5-b292-461128758c3d None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:48.969626 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac205bf3-fab8-4da5-b292-461128758c3d None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:48.971167 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 146/270] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:25:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:25:48.978034 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd01ab6-6ff5-4fd0-a8ed-00b1c284fe9a None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:48.978551 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd01ab6-6ff5-4fd0-a8ed-00b1c284fe9a None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:48.979048 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd01ab6-6ff5-4fd0-a8ed-00b1c284fe9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:48.998389 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-4bd01ab6-6ff5-4fd0-a8ed-00b1c284fe9a None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:25:48.999950 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-4bd01ab6-6ff5-4fd0-a8ed-00b1c284fe9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:25:49.052624 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e50e07d-9609-4d90-928d-b1c597e54c39 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:49.052927 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e50e07d-9609-4d90-928d-b1c597e54c39 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:49.053189 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e50e07d-9609-4d90-928d-b1c597e54c39 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:49.054564 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 147/271] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:25:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:25:49.061360 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d753cad-93ac-408e-9c37-f43714c6d1a1 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:49.061640 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d753cad-93ac-408e-9c37-f43714c6d1a1 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:49.061948 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d753cad-93ac-408e-9c37-f43714c6d1a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:49.084168 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-8d753cad-93ac-408e-9c37-f43714c6d1a1 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:25:49.084670 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 125/272] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:25:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 22:25:49.085932 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-8d753cad-93ac-408e-9c37-f43714c6d1a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:25:49.119920 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd01ab6-6ff5-4fd0-a8ed-00b1c284fe9a None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:49.173892 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 148/273] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:25:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 22:25:49.175095 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5149d33-9d41-474c-a5b5-72a71f05392e None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:49.175396 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5149d33-9d41-474c-a5b5-72a71f05392e None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:49.175682 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5149d33-9d41-474c-a5b5-72a71f05392e None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:49.177208 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 149/274] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:25:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:25:49.183260 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7c3650-2e8c-4f2e-bcee-8521bb8a1a5f None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:49.183556 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7c3650-2e8c-4f2e-bcee-8521bb8a1a5f None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:49.183846 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7c3650-2e8c-4f2e-bcee-8521bb8a1a5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:49.197601 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:49.202453 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-3f7c3650-2e8c-4f2e-bcee-8521bb8a1a5f None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:25:49.203759 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-3f7c3650-2e8c-4f2e-bcee-8521bb8a1a5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:25:49.257431 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd01ab6-6ff5-4fd0-a8ed-00b1c284fe9a None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:49.277227 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 150/275] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:25:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 22:25:49.287190 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7c3650-2e8c-4f2e-bcee-8521bb8a1a5f None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:49.288577 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:49.308157 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d087c352-6bd3-4a1f-94d2-5a9c241319c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:49.309205 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d087c352-6bd3-4a1f-94d2-5a9c241319c2 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:49.309809 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d087c352-6bd3-4a1f-94d2-5a9c241319c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:49.310096 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d087c352-6bd3-4a1f-94d2-5a9c241319c2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:49.311594 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d087c352-6bd3-4a1f-94d2-5a9c241319c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:49.313059 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d087c352-6bd3-4a1f-94d2-5a9c241319c2 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:49.319193 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 126/276] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Jul 20 22:25:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:25:49.319193 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d087c352-6bd3-4a1f-94d2-5a9c241319c2 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:49.393449 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7c3650-2e8c-4f2e-bcee-8521bb8a1a5f None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:49.424467 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d087c352-6bd3-4a1f-94d2-5a9c241319c2 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:49.431753 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf22d94-6bb8-42a0-9208-a23f91534856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:49.432567 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf22d94-6bb8-42a0-9208-a23f91534856 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:49.432855 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf22d94-6bb8-42a0-9208-a23f91534856 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:49.433140 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf22d94-6bb8-42a0-9208-a23f91534856 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:49.442602 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:49.444902 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf22d94-6bb8-42a0-9208-a23f91534856 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 22:25:49.453672 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-6cf22d94-6bb8-42a0-9208-a23f91534856 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 22:25:49.453672 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:25:49.453672 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:25:49.453672 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:25:49.453672 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:25:49.453672 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:25:49.453672 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:25:49.453672 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:25:49.453672 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:25:49.453672 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:25:49.453672 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:25:49.453672 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:25:49.453672 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 22:25:49.453672 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 22:25:49.453672 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 22:25:49.453672 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 22:25:49.460505 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 22:25:49.460505 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 22:25:49.460505 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 22:25:49.460505 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 22:25:49.460505 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:25:49.460505 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:25:49.460505 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:25:49.460505 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:25:49.460505 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:25:49.460505 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:25:49.460505 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:25:49.460505 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:25:49.460505 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:25:49.460505 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:25:49.460505 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:25:49.460505 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:25:49.461442 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:25:49.461442 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:25:49.461442 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:25:49.461442 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:25:49.461442 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 22:25:49.461442 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 22:25:49.461442 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 22:25:49.461442 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 22:25:49.461442 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 22:25:49.461442 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 22:25:49.461442 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 22:25:49.461442 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:25:49.461442 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 151/277] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Jul 20 22:25:49 2022] GET /identity/v3/roles/reader => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:25:49.461442 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf22d94-6bb8-42a0-9208-a23f91534856 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:49.471352 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d53483-db56-49d2-b099-a6aafddd8940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:49.471995 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d53483-db56-49d2-b099-a6aafddd8940 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:49.472332 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d53483-db56-49d2-b099-a6aafddd8940 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:49.472608 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d53483-db56-49d2-b099-a6aafddd8940 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:49.484772 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d53483-db56-49d2-b099-a6aafddd8940 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 22:25:49.493569 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-c0d53483-db56-49d2-b099-a6aafddd8940 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 22:25:49.493569 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:25:49.493569 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:25:49.493569 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:25:49.493569 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:25:49.493569 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:25:49.493569 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:25:49.493569 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:25:49.493569 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:25:49.493569 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:25:49.493569 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:25:49.493569 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:25:49.493569 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 22:25:49.493569 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 22:25:49.493569 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 22:25:49.493569 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 22:25:49.494448 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 22:25:49.494448 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 22:25:49.494448 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 22:25:49.494448 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 22:25:49.494448 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:25:49.494448 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:25:49.494448 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:25:49.494448 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:25:49.494448 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:25:49.494448 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:25:49.494448 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:25:49.494448 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:25:49.494448 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:25:49.494448 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:25:49.494448 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:25:49.494448 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:25:49.495279 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:25:49.495279 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:25:49.495279 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:25:49.495279 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:25:49.495279 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 22:25:49.495279 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 22:25:49.495279 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 22:25:49.495279 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 22:25:49.495279 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 22:25:49.495279 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 22:25:49.495279 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 22:25:49.495279 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:25:49.495279 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 127/278] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Jul 20 22:25:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:25:49.495925 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d53483-db56-49d2-b099-a6aafddd8940 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:49.534898 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:49.567828 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf22d94-6bb8-42a0-9208-a23f91534856 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:49.604395 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d53483-db56-49d2-b099-a6aafddd8940 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:49.604712 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafbcdb2-67ac-47fd-b16d-0210e79dbcae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:49.605357 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafbcdb2-67ac-47fd-b16d-0210e79dbcae None admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:49.605648 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafbcdb2-67ac-47fd-b16d-0210e79dbcae None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:49.605934 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafbcdb2-67ac-47fd-b16d-0210e79dbcae None admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:49.606653 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafbcdb2-67ac-47fd-b16d-0210e79dbcae None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:49.608139 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafbcdb2-67ac-47fd-b16d-0210e79dbcae None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:49.639237 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab5c93d-b858-4bc9-b195-ab95cf5bfd66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:49.639912 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab5c93d-b858-4bc9-b195-ab95cf5bfd66 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:49.640199 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab5c93d-b858-4bc9-b195-ab95cf5bfd66 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:49.640477 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab5c93d-b858-4bc9-b195-ab95cf5bfd66 None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:49.641285 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab5c93d-b858-4bc9-b195-ab95cf5bfd66 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:25:49.641593 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab5c93d-b858-4bc9-b195-ab95cf5bfd66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:49.642963 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab5c93d-b858-4bc9-b195-ab95cf5bfd66 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:49.652599 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 128/279] 10.222.0.81 () {56 vars in 1101 bytes} [Wed Jul 20 22:25:49 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:25:49.653387 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab5c93d-b858-4bc9-b195-ab95cf5bfd66 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:49.687384 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:49.722455 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 152/280] 10.222.0.81 () {60 vars in 1132 bytes} [Wed Jul 20 22:25:49 2022] POST /identity/v3/users => generated 293 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:25:49.723595 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafbcdb2-67ac-47fd-b16d-0210e79dbcae None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:49.763356 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab5c93d-b858-4bc9-b195-ab95cf5bfd66 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:49.804219 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbbe18b-aea0-4969-9c6d-6cdd3122ed43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:49.804830 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbbe18b-aea0-4969-9c6d-6cdd3122ed43 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:49.805121 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbbe18b-aea0-4969-9c6d-6cdd3122ed43 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:49.805428 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbbe18b-aea0-4969-9c6d-6cdd3122ed43 None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:49.806249 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbbe18b-aea0-4969-9c6d-6cdd3122ed43 None admin] RBAC: Adding query filter params (name=member) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:25:49.806564 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbbe18b-aea0-4969-9c6d-6cdd3122ed43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:49.808054 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbbe18b-aea0-4969-9c6d-6cdd3122ed43 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:49.817693 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 129/281] 10.222.0.81 () {56 vars in 1101 bytes} [Wed Jul 20 22:25:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:25:49.820550 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63d71db-e39d-4c2b-b7f6-d394f9fee648 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:49.820853 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63d71db-e39d-4c2b-b7f6-d394f9fee648 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:49.823419 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63d71db-e39d-4c2b-b7f6-d394f9fee648 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:49.823419 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 130/282] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:25:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:25:49.824007 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63d71db-e39d-4c2b-b7f6-d394f9fee648 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:49.846045 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafbcdb2-67ac-47fd-b16d-0210e79dbcae None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:49.872810 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:49.885173 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ada53c-9bed-4f56-b497-b201e395e8a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:49.887095 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ada53c-9bed-4f56-b497-b201e395e8a5 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:49.887095 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ada53c-9bed-4f56-b497-b201e395e8a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:49.887095 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ada53c-9bed-4f56-b497-b201e395e8a5 None admin] PATH_INFO: `/v3/users/6e399a52b2b746e4978ef90c96c96594` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:49.888623 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ada53c-9bed-4f56-b497-b201e395e8a5 None admin] RBAC: Authorizing `identity:get_user(user_id=6e399a52b2b746e4978ef90c96c96594)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:49.890084 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ada53c-9bed-4f56-b497-b201e395e8a5 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:49.896388 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 153/283] 10.222.0.81 () {56 vars in 1177 bytes} [Wed Jul 20 22:25:49 2022] GET /identity/v3/users/6e399a52b2b746e4978ef90c96c96594 => generated 297 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:25:49.896388 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd708d5-449d-4226-ba77-b2ba1dfa3fce None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:49.896388 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd708d5-449d-4226-ba77-b2ba1dfa3fce None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:49.896388 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd708d5-449d-4226-ba77-b2ba1dfa3fce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:49.912172 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-1bd708d5-449d-4226-ba77-b2ba1dfa3fce None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:25:49.913339 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-1bd708d5-449d-4226-ba77-b2ba1dfa3fce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:25:49.949000 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:49.954164 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63d71db-e39d-4c2b-b7f6-d394f9fee648 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:49.995464 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 154/284] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:25:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 22:25:49.996182 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd708d5-449d-4226-ba77-b2ba1dfa3fce None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:50.002042 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a15435-bd07-4ed1-bc05-bb878c48a0eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:50.003313 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a15435-bd07-4ed1-bc05-bb878c48a0eb None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:50.003799 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a15435-bd07-4ed1-bc05-bb878c48a0eb None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:50.004268 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a15435-bd07-4ed1-bc05-bb878c48a0eb None admin] PATH_INFO: `/v3/users/f3e955b84b7d4c9d91b15e961e4ec65e` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:50.021090 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a15435-bd07-4ed1-bc05-bb878c48a0eb None admin] RBAC: Authorizing `identity:get_user(user_id=f3e955b84b7d4c9d91b15e961e4ec65e)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:50.023656 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a15435-bd07-4ed1-bc05-bb878c48a0eb None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:50.026865 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 131/285] 10.222.0.81 () {56 vars in 1177 bytes} [Wed Jul 20 22:25:49 2022] GET /identity/v3/users/f3e955b84b7d4c9d91b15e961e4ec65e => generated 293 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:25:50.028172 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a15435-bd07-4ed1-bc05-bb878c48a0eb None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:50.133241 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd708d5-449d-4226-ba77-b2ba1dfa3fce None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:50.145337 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a15435-bd07-4ed1-bc05-bb878c48a0eb None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:50.164280 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:50.182217 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4307427-f33e-4778-9f19-7b20970267dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:50.183659 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4307427-f33e-4778-9f19-7b20970267dd None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:50.183946 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4307427-f33e-4778-9f19-7b20970267dd None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:50.190809 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4307427-f33e-4778-9f19-7b20970267dd None admin] PATH_INFO: `/v3/projects/7165d638ee164d27903602ef884fcf03` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:50.190809 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4307427-f33e-4778-9f19-7b20970267dd None admin] RBAC: Authorizing `identity:get_project(project_id=7165d638ee164d27903602ef884fcf03)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:50.190997 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aee4ed7-a111-4bfd-a021-00013490cf38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:50.191107 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4307427-f33e-4778-9f19-7b20970267dd None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:50.191177 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aee4ed7-a111-4bfd-a021-00013490cf38 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:50.191177 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aee4ed7-a111-4bfd-a021-00013490cf38 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:50.191177 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aee4ed7-a111-4bfd-a021-00013490cf38 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:50.191359 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 155/286] 10.222.0.81 () {56 vars in 1186 bytes} [Wed Jul 20 22:25:49 2022] GET /identity/v3/projects/7165d638ee164d27903602ef884fcf03 => generated 302 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:25:50.191359 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4307427-f33e-4778-9f19-7b20970267dd None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:50.212884 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aee4ed7-a111-4bfd-a021-00013490cf38 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 22:25:50.222352 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-0aee4ed7-a111-4bfd-a021-00013490cf38 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 22:25:50.222352 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:25:50.222352 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:25:50.222352 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:25:50.222352 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:25:50.222352 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:25:50.222352 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:25:50.222352 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:25:50.222352 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:25:50.222352 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:25:50.222352 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:25:50.222352 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:25:50.222352 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 22:25:50.222352 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 22:25:50.222352 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 22:25:50.222352 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 22:25:50.223277 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 22:25:50.223277 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 22:25:50.223277 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 22:25:50.223277 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 22:25:50.223277 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:25:50.223277 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:25:50.223277 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:25:50.223277 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:25:50.223277 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:25:50.223277 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:25:50.223277 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:25:50.223277 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:25:50.223277 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:25:50.223277 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:25:50.223277 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:25:50.223277 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:25:50.224080 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:25:50.224080 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:25:50.224080 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:25:50.224080 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:25:50.224080 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 22:25:50.224080 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 22:25:50.224080 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 22:25:50.224080 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 22:25:50.224080 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 22:25:50.224080 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 22:25:50.224080 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 22:25:50.224080 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:25:50.224080 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 132/287] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Jul 20 22:25:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:25:50.224717 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aee4ed7-a111-4bfd-a021-00013490cf38 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:50.247632 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:50.322016 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4307427-f33e-4778-9f19-7b20970267dd None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:50.349136 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aee4ed7-a111-4bfd-a021-00013490cf38 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:50.359894 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1f9d5d-a340-471f-b1c4-b47d4b2b0f56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:50.360561 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1f9d5d-a340-471f-b1c4-b47d4b2b0f56 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:50.360850 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1f9d5d-a340-471f-b1c4-b47d4b2b0f56 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:50.361133 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1f9d5d-a340-471f-b1c4-b47d4b2b0f56 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:50.362287 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1f9d5d-a340-471f-b1c4-b47d4b2b0f56 None admin] RBAC: Adding query filter params (role.id=559942c4de1744169f8b5e0404ecde6b, user.id=f3e955b84b7d4c9d91b15e961e4ec65e, scope.system=all) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:25:50.362395 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1f9d5d-a340-471f-b1c4-b47d4b2b0f56 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:50.364132 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1f9d5d-a340-471f-b1c4-b47d4b2b0f56 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:50.369783 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 156/288] 10.222.0.81 () {56 vars in 1308 bytes} [Wed Jul 20 22:25:50 2022] GET /identity/v3/role_assignments?user.id=f3e955b84b7d4c9d91b15e961e4ec65e&scope.system=all&role.id=559942c4de1744169f8b5e0404ecde6b => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:25:50.370502 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1f9d5d-a340-471f-b1c4-b47d4b2b0f56 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:50.393622 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f1bb07-5f01-4482-a8f9-39a0b80088e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:50.394503 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f1bb07-5f01-4482-a8f9-39a0b80088e8 None admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:50.394793 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f1bb07-5f01-4482-a8f9-39a0b80088e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:50.395429 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f1bb07-5f01-4482-a8f9-39a0b80088e8 None admin] PATH_INFO: `/v3/projects/7165d638ee164d27903602ef884fcf03/users/6e399a52b2b746e4978ef90c96c96594/roles/5b2ef0feadf94c059dffbd7d8bc03aa4` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:50.402653 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f1bb07-5f01-4482-a8f9-39a0b80088e8 None admin] RBAC: Authorizing `identity:create_grant(project_id=7165d638ee164d27903602ef884fcf03, user_id=6e399a52b2b746e4978ef90c96c96594, role_id=5b2ef0feadf94c059dffbd7d8bc03aa4)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:50.402653 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f1bb07-5f01-4482-a8f9-39a0b80088e8 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:50.447574 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:50.497373 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 133/289] 10.222.0.81 () {58 vars in 1439 bytes} [Wed Jul 20 22:25:50 2022] PUT /identity/v3/projects/7165d638ee164d27903602ef884fcf03/users/6e399a52b2b746e4978ef90c96c96594/roles/5b2ef0feadf94c059dffbd7d8bc03aa4 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:25:50.498835 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1f9d5d-a340-471f-b1c4-b47d4b2b0f56 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:50.546684 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d356420-29fa-41ad-9e27-299b3cd0f144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:50.547807 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d356420-29fa-41ad-9e27-299b3cd0f144 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:50.548299 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d356420-29fa-41ad-9e27-299b3cd0f144 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:50.548845 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d356420-29fa-41ad-9e27-299b3cd0f144 None admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:50.549827 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d356420-29fa-41ad-9e27-299b3cd0f144 None admin] RBAC: Adding query filter params (name=member) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:25:50.550371 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d356420-29fa-41ad-9e27-299b3cd0f144 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:50.552020 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d356420-29fa-41ad-9e27-299b3cd0f144 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:50.562556 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 157/290] 10.222.0.81 () {56 vars in 1101 bytes} [Wed Jul 20 22:25:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:25:50.568308 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f1bb07-5f01-4482-a8f9-39a0b80088e8 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:50.680583 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:50.705997 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f1bb07-5f01-4482-a8f9-39a0b80088e8 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:50.742056 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0c4dac-6505-4751-9b65-a0ea0292a149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:50.743538 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0c4dac-6505-4751-9b65-a0ea0292a149 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:50.743831 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0c4dac-6505-4751-9b65-a0ea0292a149 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:50.744115 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0c4dac-6505-4751-9b65-a0ea0292a149 None admin] PATH_INFO: `/v3/users/56686220e81f4756a9ea2d7d411d7e32` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:50.745351 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0c4dac-6505-4751-9b65-a0ea0292a149 None admin] RBAC: Authorizing `identity:get_user(user_id=56686220e81f4756a9ea2d7d411d7e32)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:50.746805 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0c4dac-6505-4751-9b65-a0ea0292a149 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:50.748892 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 134/291] 10.222.0.81 () {56 vars in 1177 bytes} [Wed Jul 20 22:25:50 2022] GET /identity/v3/users/56686220e81f4756a9ea2d7d411d7e32 => generated 297 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:25:50.754411 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d356420-29fa-41ad-9e27-299b3cd0f144 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:50.866920 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d356420-29fa-41ad-9e27-299b3cd0f144 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:50.891259 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:50.913697 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062539c2-d368-49d8-993b-08681c421aa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:50.914411 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062539c2-d368-49d8-993b-08681c421aa3 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:50.914754 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062539c2-d368-49d8-993b-08681c421aa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:50.915568 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062539c2-d368-49d8-993b-08681c421aa3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:51.008478 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062539c2-d368-49d8-993b-08681c421aa3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:51.009864 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062539c2-d368-49d8-993b-08681c421aa3 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:51.145526 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 158/292] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Jul 20 22:25:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 22:25:51.151799 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0c4dac-6505-4751-9b65-a0ea0292a149 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:51.303602 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0c4dac-6505-4751-9b65-a0ea0292a149 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:51.352056 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177aaa43-736e-43da-89e0-af727ca882df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:51.353759 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177aaa43-736e-43da-89e0-af727ca882df None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:51.354225 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177aaa43-736e-43da-89e0-af727ca882df None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:51.354681 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177aaa43-736e-43da-89e0-af727ca882df None admin] PATH_INFO: `/v3/projects/7165d638ee164d27903602ef884fcf03` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:51.356087 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177aaa43-736e-43da-89e0-af727ca882df None admin] RBAC: Authorizing `identity:get_project(project_id=7165d638ee164d27903602ef884fcf03)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:51.357733 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177aaa43-736e-43da-89e0-af727ca882df None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:51.359668 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 135/293] 10.222.0.81 () {56 vars in 1186 bytes} [Wed Jul 20 22:25:51 2022] GET /identity/v3/projects/7165d638ee164d27903602ef884fcf03 => generated 302 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:25:51.365736 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062539c2-d368-49d8-993b-08681c421aa3 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:51.394663 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:51.654684 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062539c2-d368-49d8-993b-08681c421aa3 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:51.745295 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c84679-6f88-4e66-8eaa-8daa484ad561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:51.745967 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c84679-6f88-4e66-8eaa-8daa484ad561 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:51.746256 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c84679-6f88-4e66-8eaa-8daa484ad561 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:51.746540 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c84679-6f88-4e66-8eaa-8daa484ad561 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:51.747414 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c84679-6f88-4e66-8eaa-8daa484ad561 None admin] RBAC: Adding query filter params (role.id=559942c4de1744169f8b5e0404ecde6b, scope.project.id=7165d638ee164d27903602ef884fcf03, user.id=56686220e81f4756a9ea2d7d411d7e32) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:25:51.747960 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c84679-6f88-4e66-8eaa-8daa484ad561 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:51.749598 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c84679-6f88-4e66-8eaa-8daa484ad561 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:51.756289 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 159/294] 10.222.0.81 () {56 vars in 1374 bytes} [Wed Jul 20 22:25:51 2022] GET /identity/v3/role_assignments?user.id=56686220e81f4756a9ea2d7d411d7e32&scope.project.id=7165d638ee164d27903602ef884fcf03&role.id=559942c4de1744169f8b5e0404ecde6b => generated 608 bytes in 391 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:25:52.359809 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccf4193-4504-479a-a17c-642c6319144b None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:52.359809 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccf4193-4504-479a-a17c-642c6319144b None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:52.359809 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccf4193-4504-479a-a17c-642c6319144b None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:52.360827 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 136/295] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:25:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:25:52.368175 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123e01fb-8b46-4f56-930a-3a330939a2e5 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:52.369003 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123e01fb-8b46-4f56-930a-3a330939a2e5 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:52.369504 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123e01fb-8b46-4f56-930a-3a330939a2e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:52.386934 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-123e01fb-8b46-4f56-930a-3a330939a2e5 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:25:52.388453 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-123e01fb-8b46-4f56-930a-3a330939a2e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:25:52.455200 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc611fe-634d-4048-9087-242425301146 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:52.455790 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc611fe-634d-4048-9087-242425301146 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:52.456279 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc611fe-634d-4048-9087-242425301146 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:52.457882 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 137/296] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:25:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:25:52.460493 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 160/297] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:25:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 22:25:52.465593 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a08716e-900e-43a8-8010-e826b5b1bc24 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:52.466126 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a08716e-900e-43a8-8010-e826b5b1bc24 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:52.466639 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a08716e-900e-43a8-8010-e826b5b1bc24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:52.485022 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123e01fb-8b46-4f56-930a-3a330939a2e5 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:52.487758 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-2a08716e-900e-43a8-8010-e826b5b1bc24 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:25:52.489197 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-2a08716e-900e-43a8-8010-e826b5b1bc24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:25:52.498357 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:52.498499 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:52.566498 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 138/298] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:25:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 22:25:52.596071 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a08716e-900e-43a8-8010-e826b5b1bc24 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:52.610286 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123e01fb-8b46-4f56-930a-3a330939a2e5 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:52.654349 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7e79df-eadc-4250-a803-c29c69862c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:52.655616 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7e79df-eadc-4250-a803-c29c69862c55 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:52.656254 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7e79df-eadc-4250-a803-c29c69862c55 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:52.656846 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7e79df-eadc-4250-a803-c29c69862c55 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:52.658245 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7e79df-eadc-4250-a803-c29c69862c55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:52.660139 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7e79df-eadc-4250-a803-c29c69862c55 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:52.663675 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 161/299] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Jul 20 22:25:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:25:52.669828 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7e79df-eadc-4250-a803-c29c69862c55 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:52.720124 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a08716e-900e-43a8-8010-e826b5b1bc24 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:52.763439 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1255407f-630c-4e2c-8480-6addfda087c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:52.765087 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1255407f-630c-4e2c-8480-6addfda087c4 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:52.765942 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1255407f-630c-4e2c-8480-6addfda087c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:52.766435 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1255407f-630c-4e2c-8480-6addfda087c4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:52.767284 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:52.781020 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1255407f-630c-4e2c-8480-6addfda087c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 22:25:52.791284 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7e79df-eadc-4250-a803-c29c69862c55 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:52.792530 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-1255407f-630c-4e2c-8480-6addfda087c4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 22:25:52.792530 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:25:52.792530 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:25:52.792530 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:25:52.792530 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:25:52.792530 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:25:52.792530 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:25:52.792530 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:25:52.792530 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:25:52.792530 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:25:52.792530 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:25:52.792530 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:25:52.792530 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 22:25:52.792530 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 22:25:52.792530 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 22:25:52.792530 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 22:25:52.793661 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 22:25:52.793661 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 22:25:52.793661 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 22:25:52.793661 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 22:25:52.793661 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:25:52.793661 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:25:52.793661 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:25:52.793661 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:25:52.793661 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:25:52.793661 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:25:52.793661 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:25:52.793661 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:25:52.793661 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:25:52.793661 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:25:52.793661 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:25:52.793661 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:25:52.794444 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:25:52.794444 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:25:52.794444 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:25:52.794444 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:25:52.794444 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 22:25:52.794444 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 22:25:52.794444 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 22:25:52.794444 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 22:25:52.794444 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 22:25:52.794444 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 22:25:52.794444 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 22:25:52.794444 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:25:52.799364 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 139/300] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Jul 20 22:25:52 2022] GET /identity/v3/roles/reader => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 22:25:52.802982 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1255407f-630c-4e2c-8480-6addfda087c4 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:52.806312 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:52.843622 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d03d36-b19c-4505-bf7b-8befa0bc196e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:52.844441 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d03d36-b19c-4505-bf7b-8befa0bc196e None admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:52.844728 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d03d36-b19c-4505-bf7b-8befa0bc196e None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:52.845004 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d03d36-b19c-4505-bf7b-8befa0bc196e None admin] PATH_INFO: `/v3/groups` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:52.845702 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d03d36-b19c-4505-bf7b-8befa0bc196e None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:52.847980 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d03d36-b19c-4505-bf7b-8befa0bc196e None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:52.907726 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 162/301] 10.222.0.81 () {60 vars in 1134 bytes} [Wed Jul 20 22:25:52 2022] POST /identity/v3/groups => generated 229 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 22:25:52.929046 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1255407f-630c-4e2c-8480-6addfda087c4 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:52.938980 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1df4252-269b-47c9-8ee6-8a4c917e66a4 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:52.939388 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1df4252-269b-47c9-8ee6-8a4c917e66a4 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:52.939601 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1df4252-269b-47c9-8ee6-8a4c917e66a4 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:52.941020 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 163/302] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:25:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 22:25:52.947178 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7147ed-10c3-456c-bf5c-08a103b2df49 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:52.947602 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7147ed-10c3-456c-bf5c-08a103b2df49 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:52.947735 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7147ed-10c3-456c-bf5c-08a103b2df49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:52.966035 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-ef7147ed-10c3-456c-bf5c-08a103b2df49 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:25:52.967515 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-ef7147ed-10c3-456c-bf5c-08a103b2df49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:25:52.977026 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d1e4ae-f38f-48b7-b19b-f5dc2d877434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:52.978675 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d1e4ae-f38f-48b7-b19b-f5dc2d877434 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:52.979387 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d1e4ae-f38f-48b7-b19b-f5dc2d877434 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:52.979819 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d1e4ae-f38f-48b7-b19b-f5dc2d877434 None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:52.980969 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d1e4ae-f38f-48b7-b19b-f5dc2d877434 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:25:52.981294 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d1e4ae-f38f-48b7-b19b-f5dc2d877434 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:52.982770 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d1e4ae-f38f-48b7-b19b-f5dc2d877434 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:52.994068 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 140/303] 10.222.0.81 () {56 vars in 1101 bytes} [Wed Jul 20 22:25:52 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 22:25:52.999654 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d1e4ae-f38f-48b7-b19b-f5dc2d877434 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:53.056778 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 164/304] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:25:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 22:25:53.082566 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7147ed-10c3-456c-bf5c-08a103b2df49 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:53.090868 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:53.136728 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d1e4ae-f38f-48b7-b19b-f5dc2d877434 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:53.163846 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:53.188698 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fe091a-42a4-4fe7-a4dd-7a17d4be46da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:53.190109 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fe091a-42a4-4fe7-a4dd-7a17d4be46da None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:53.190396 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fe091a-42a4-4fe7-a4dd-7a17d4be46da None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:53.190672 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fe091a-42a4-4fe7-a4dd-7a17d4be46da None admin] PATH_INFO: `/v3/users/9bf09b5a20154dc7a8a0645b0f9a9d97` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:53.204678 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7147ed-10c3-456c-bf5c-08a103b2df49 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:53.207985 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fe091a-42a4-4fe7-a4dd-7a17d4be46da None admin] RBAC: Authorizing `identity:get_user(user_id=9bf09b5a20154dc7a8a0645b0f9a9d97)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:53.209667 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fe091a-42a4-4fe7-a4dd-7a17d4be46da None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:53.216431 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 141/305] 10.222.0.81 () {56 vars in 1177 bytes} [Wed Jul 20 22:25:52 2022] GET /identity/v3/users/9bf09b5a20154dc7a8a0645b0f9a9d97 => generated 293 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 22:25:53.216431 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5a251f-94d7-4730-9069-5d3f00b6a723 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:53.216431 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5a251f-94d7-4730-9069-5d3f00b6a723 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:53.216431 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5a251f-94d7-4730-9069-5d3f00b6a723 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:53.216431 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 142/306] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:25:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:25:53.218314 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5a251f-94d7-4730-9069-5d3f00b6a723 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:53.248610 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98896a9c-171d-4e56-b8fe-7d73fca59d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:53.249346 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98896a9c-171d-4e56-b8fe-7d73fca59d44 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:53.249634 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98896a9c-171d-4e56-b8fe-7d73fca59d44 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:53.249912 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98896a9c-171d-4e56-b8fe-7d73fca59d44 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:53.261377 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98896a9c-171d-4e56-b8fe-7d73fca59d44 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 22:25:53.270730 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-98896a9c-171d-4e56-b8fe-7d73fca59d44 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 22:25:53.270730 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:25:53.270730 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:25:53.270730 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:25:53.270730 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:25:53.270730 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:25:53.270730 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:25:53.270730 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:25:53.270730 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:25:53.270730 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:25:53.270730 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:25:53.270730 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:25:53.270730 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 22:25:53.270730 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 22:25:53.270730 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 22:25:53.270730 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 22:25:53.271669 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 22:25:53.271669 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 22:25:53.271669 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 22:25:53.271669 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 22:25:53.271669 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:25:53.271669 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:25:53.271669 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:25:53.271669 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:25:53.271669 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:25:53.271669 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:25:53.271669 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:25:53.271669 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:25:53.271669 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:25:53.271669 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:25:53.271669 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:25:53.271669 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:25:53.272455 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:25:53.272455 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:25:53.272455 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:25:53.272455 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:25:53.272455 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 22:25:53.272455 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 22:25:53.272455 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 22:25:53.272455 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 22:25:53.272455 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 22:25:53.272455 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 22:25:53.272455 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 22:25:53.272455 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:25:53.272455 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 165/307] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Jul 20 22:25:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 22:25:53.277627 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ad5332-7c28-4953-8085-ae148f71d49a None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:53.277912 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ad5332-7c28-4953-8085-ae148f71d49a None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:53.278190 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ad5332-7c28-4953-8085-ae148f71d49a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:53.308305 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-32ad5332-7c28-4953-8085-ae148f71d49a None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:25:53.310399 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-32ad5332-7c28-4953-8085-ae148f71d49a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:25:53.340691 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5a251f-94d7-4730-9069-5d3f00b6a723 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:53.366596 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:53.380303 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b38128-ae99-4cf6-9a16-d3dda421d054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:53.380977 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b38128-ae99-4cf6-9a16-d3dda421d054 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:53.381266 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b38128-ae99-4cf6-9a16-d3dda421d054 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:53.381545 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b38128-ae99-4cf6-9a16-d3dda421d054 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:53.382505 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b38128-ae99-4cf6-9a16-d3dda421d054 None admin] RBAC: Adding query filter params (role.id=5b2ef0feadf94c059dffbd7d8bc03aa4, user.id=9bf09b5a20154dc7a8a0645b0f9a9d97, scope.system=all) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:25:53.382935 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b38128-ae99-4cf6-9a16-d3dda421d054 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:53.384969 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b38128-ae99-4cf6-9a16-d3dda421d054 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:53.399218 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 143/308] 10.222.0.81 () {56 vars in 1308 bytes} [Wed Jul 20 22:25:53 2022] GET /identity/v3/role_assignments?user.id=9bf09b5a20154dc7a8a0645b0f9a9d97&scope.system=all&role.id=5b2ef0feadf94c059dffbd7d8bc03aa4 => generated 226 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:25:53.399218 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b38128-ae99-4cf6-9a16-d3dda421d054 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:53.401600 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 166/309] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:25:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 22:25:53.425777 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ad5332-7c28-4953-8085-ae148f71d49a None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:53.487901 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:53.515732 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b38128-ae99-4cf6-9a16-d3dda421d054 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:53.545867 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ad5332-7c28-4953-8085-ae148f71d49a None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:53.554545 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b35ae3-20bb-4d8a-9377-7b00682099e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:53.555850 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b35ae3-20bb-4d8a-9377-7b00682099e2 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:53.556260 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b35ae3-20bb-4d8a-9377-7b00682099e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:53.556554 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b35ae3-20bb-4d8a-9377-7b00682099e2 None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:53.557317 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b35ae3-20bb-4d8a-9377-7b00682099e2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:25:53.557630 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b35ae3-20bb-4d8a-9377-7b00682099e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:53.559237 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b35ae3-20bb-4d8a-9377-7b00682099e2 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:53.574389 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 144/310] 10.222.0.81 () {56 vars in 1101 bytes} [Wed Jul 20 22:25:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:25:53.575819 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b35ae3-20bb-4d8a-9377-7b00682099e2 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:53.591698 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14ffd3b-86b4-4016-a36c-dd7a045b9f89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:53.592817 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14ffd3b-86b4-4016-a36c-dd7a045b9f89 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:53.593252 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14ffd3b-86b4-4016-a36c-dd7a045b9f89 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:53.593782 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14ffd3b-86b4-4016-a36c-dd7a045b9f89 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:53.607703 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14ffd3b-86b4-4016-a36c-dd7a045b9f89 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 22:25:53.616044 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-a14ffd3b-86b4-4016-a36c-dd7a045b9f89 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 22:25:53.616044 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:25:53.616044 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:25:53.616044 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:25:53.616044 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:25:53.616044 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:25:53.616044 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:25:53.616044 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:25:53.616044 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:25:53.616044 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:25:53.616044 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:25:53.616044 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:25:53.616044 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 22:25:53.616044 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 22:25:53.616044 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 22:25:53.616044 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 22:25:53.617089 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 22:25:53.617089 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 22:25:53.617089 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 22:25:53.617089 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 22:25:53.617089 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:25:53.617089 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:25:53.617089 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:25:53.617089 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:25:53.617089 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:25:53.617089 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:25:53.617089 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:25:53.617089 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:25:53.617089 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:25:53.617089 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:25:53.617089 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:25:53.617089 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:25:53.618029 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:25:53.618029 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:25:53.618029 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:25:53.618029 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:25:53.618029 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 22:25:53.618029 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 22:25:53.618029 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 22:25:53.618029 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 22:25:53.618029 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 22:25:53.618029 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 22:25:53.618029 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 22:25:53.618029 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:25:53.618029 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 167/311] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Jul 20 22:25:53 2022] GET /identity/v3/roles/reader => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:25:53.623296 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14ffd3b-86b4-4016-a36c-dd7a045b9f89 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:53.654454 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:53.711082 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b35ae3-20bb-4d8a-9377-7b00682099e2 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:53.739817 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14ffd3b-86b4-4016-a36c-dd7a045b9f89 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:53.747186 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb40cd7-5c9d-403d-a323-b66b361f8a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:53.748361 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb40cd7-5c9d-403d-a323-b66b361f8a32 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:53.748794 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb40cd7-5c9d-403d-a323-b66b361f8a32 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:53.749212 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb40cd7-5c9d-403d-a323-b66b361f8a32 None admin] PATH_INFO: `/v3/users/f3e955b84b7d4c9d91b15e961e4ec65e` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:53.750584 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb40cd7-5c9d-403d-a323-b66b361f8a32 None admin] RBAC: Authorizing `identity:get_user(user_id=f3e955b84b7d4c9d91b15e961e4ec65e)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:53.752224 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb40cd7-5c9d-403d-a323-b66b361f8a32 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:53.754008 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 145/312] 10.222.0.81 () {56 vars in 1177 bytes} [Wed Jul 20 22:25:53 2022] GET /identity/v3/users/f3e955b84b7d4c9d91b15e961e4ec65e => generated 293 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:25:53.761205 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb40cd7-5c9d-403d-a323-b66b361f8a32 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:53.764934 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:53.783723 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eefc67f-93ec-491d-baa6-142b81aec779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:53.784332 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eefc67f-93ec-491d-baa6-142b81aec779 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:53.784624 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eefc67f-93ec-491d-baa6-142b81aec779 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:53.784909 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eefc67f-93ec-491d-baa6-142b81aec779 None admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:53.786041 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eefc67f-93ec-491d-baa6-142b81aec779 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:25:53.786156 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eefc67f-93ec-491d-baa6-142b81aec779 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:53.787558 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eefc67f-93ec-491d-baa6-142b81aec779 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:53.802697 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 168/313] 10.222.0.81 () {56 vars in 1101 bytes} [Wed Jul 20 22:25:53 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 22:25:53.804100 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eefc67f-93ec-491d-baa6-142b81aec779 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:53.868491 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb40cd7-5c9d-403d-a323-b66b361f8a32 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:53.886215 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:53.904639 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb89b1c-43d8-4e21-a270-5881a57aa2b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:53.905762 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb89b1c-43d8-4e21-a270-5881a57aa2b3 None admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:53.906191 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb89b1c-43d8-4e21-a270-5881a57aa2b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:53.906612 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb89b1c-43d8-4e21-a270-5881a57aa2b3 None admin] PATH_INFO: `/v3/system/users/f3e955b84b7d4c9d91b15e961e4ec65e/roles/559942c4de1744169f8b5e0404ecde6b` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:53.908366 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb89b1c-43d8-4e21-a270-5881a57aa2b3 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=f3e955b84b7d4c9d91b15e961e4ec65e, role_id=559942c4de1744169f8b5e0404ecde6b)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:53.909875 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb89b1c-43d8-4e21-a270-5881a57aa2b3 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:53.914907 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eefc67f-93ec-491d-baa6-142b81aec779 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:53.952498 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674f3457-912e-4011-b981-ec82cc843662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:53.953487 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674f3457-912e-4011-b981-ec82cc843662 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:53.953780 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674f3457-912e-4011-b981-ec82cc843662 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:53.954071 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674f3457-912e-4011-b981-ec82cc843662 None admin] PATH_INFO: `/v3/users/6e399a52b2b746e4978ef90c96c96594` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:53.955307 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674f3457-912e-4011-b981-ec82cc843662 None admin] RBAC: Authorizing `identity:get_user(user_id=6e399a52b2b746e4978ef90c96c96594)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:53.956751 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674f3457-912e-4011-b981-ec82cc843662 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:53.961819 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 169/314] 10.222.0.81 () {56 vars in 1177 bytes} [Wed Jul 20 22:25:53 2022] GET /identity/v3/users/6e399a52b2b746e4978ef90c96c96594 => generated 297 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:25:53.964164 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674f3457-912e-4011-b981-ec82cc843662 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:54.009445 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 146/315] 10.222.0.81 () {58 vars in 1334 bytes} [Wed Jul 20 22:25:53 2022] PUT /identity/v3/system/users/f3e955b84b7d4c9d91b15e961e4ec65e/roles/559942c4de1744169f8b5e0404ecde6b => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:25:54.022746 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:54.081625 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674f3457-912e-4011-b981-ec82cc843662 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:54.123794 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d31f3f-2938-4ac7-a76f-186d8724a36b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:54.124889 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d31f3f-2938-4ac7-a76f-186d8724a36b None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:54.125413 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d31f3f-2938-4ac7-a76f-186d8724a36b None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:54.125923 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d31f3f-2938-4ac7-a76f-186d8724a36b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:54.212357 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d31f3f-2938-4ac7-a76f-186d8724a36b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:54.213849 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d31f3f-2938-4ac7-a76f-186d8724a36b None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:54.264122 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:54.323125 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 170/316] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Jul 20 22:25:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 20 22:25:54.325211 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb89b1c-43d8-4e21-a270-5881a57aa2b3 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:54.431237 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb89b1c-43d8-4e21-a270-5881a57aa2b3 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:54.466656 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deefebb-9cd3-4099-ab62-d91f71ff0125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:54.467626 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deefebb-9cd3-4099-ab62-d91f71ff0125 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:54.467919 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deefebb-9cd3-4099-ab62-d91f71ff0125 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:54.468203 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deefebb-9cd3-4099-ab62-d91f71ff0125 None admin] PATH_INFO: `/v3/projects/7165d638ee164d27903602ef884fcf03` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:54.469403 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0deefebb-9cd3-4099-ab62-d91f71ff0125 None admin] RBAC: Authorizing `identity:get_project(project_id=7165d638ee164d27903602ef884fcf03)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:54.470843 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0deefebb-9cd3-4099-ab62-d91f71ff0125 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:54.472758 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 147/317] 10.222.0.81 () {56 vars in 1186 bytes} [Wed Jul 20 22:25:54 2022] GET /identity/v3/projects/7165d638ee164d27903602ef884fcf03 => generated 302 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:25:54.479190 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d31f3f-2938-4ac7-a76f-186d8724a36b None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:54.585200 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d31f3f-2938-4ac7-a76f-186d8724a36b None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:54.621811 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387c8856-b4d6-4c1f-86ef-60aae4ee27ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:54.622807 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387c8856-b4d6-4c1f-86ef-60aae4ee27ff None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:54.623320 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387c8856-b4d6-4c1f-86ef-60aae4ee27ff None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:54.623914 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387c8856-b4d6-4c1f-86ef-60aae4ee27ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:54.624945 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387c8856-b4d6-4c1f-86ef-60aae4ee27ff None admin] RBAC: Adding query filter params (role.id=5b2ef0feadf94c059dffbd7d8bc03aa4, scope.project.id=7165d638ee164d27903602ef884fcf03, user.id=6e399a52b2b746e4978ef90c96c96594) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:25:54.625470 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387c8856-b4d6-4c1f-86ef-60aae4ee27ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:54.627136 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387c8856-b4d6-4c1f-86ef-60aae4ee27ff None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:54.634294 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 171/318] 10.222.0.81 () {56 vars in 1374 bytes} [Wed Jul 20 22:25:54 2022] GET /identity/v3/role_assignments?user.id=6e399a52b2b746e4978ef90c96c96594&scope.project.id=7165d638ee164d27903602ef884fcf03&role.id=5b2ef0feadf94c059dffbd7d8bc03aa4 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:25:55.046472 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8cd5b6-0faf-48fb-9991-3489581df080 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:55.046766 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8cd5b6-0faf-48fb-9991-3489581df080 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:55.047053 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8cd5b6-0faf-48fb-9991-3489581df080 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:55.048585 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 148/319] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:25:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:25:55.053569 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd46920-97c7-4c74-83d5-c3b10010d698 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:55.053863 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd46920-97c7-4c74-83d5-c3b10010d698 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:55.054148 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd46920-97c7-4c74-83d5-c3b10010d698 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:55.071378 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-dfd46920-97c7-4c74-83d5-c3b10010d698 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:25:55.072613 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-dfd46920-97c7-4c74-83d5-c3b10010d698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:25:55.080425 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:55.138491 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 172/320] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:25:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 22:25:55.166637 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8cd5b6-0faf-48fb-9991-3489581df080 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:55.222479 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:55.275675 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8cd5b6-0faf-48fb-9991-3489581df080 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:55.309362 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf2a712-d4af-46bd-b038-27a575272972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:55.310441 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf2a712-d4af-46bd-b038-27a575272972 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:55.310760 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf2a712-d4af-46bd-b038-27a575272972 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:55.311226 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf2a712-d4af-46bd-b038-27a575272972 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:55.312746 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf2a712-d4af-46bd-b038-27a575272972 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:55.314446 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf2a712-d4af-46bd-b038-27a575272972 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:55.316308 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 149/321] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Jul 20 22:25:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:25:55.322196 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd46920-97c7-4c74-83d5-c3b10010d698 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:55.424655 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd46920-97c7-4c74-83d5-c3b10010d698 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:55.459122 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c18000-0125-455b-887a-884f929bdbdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:55.459792 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c18000-0125-455b-887a-884f929bdbdc None admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:55.460080 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c18000-0125-455b-887a-884f929bdbdc None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:55.460363 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c18000-0125-455b-887a-884f929bdbdc None admin] PATH_INFO: `/v3/groups` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:55.461064 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c18000-0125-455b-887a-884f929bdbdc None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:55.462613 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c18000-0125-455b-887a-884f929bdbdc None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:55.515483 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598e0dcc-aeda-40da-a912-83cc536e502e None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:55.516005 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598e0dcc-aeda-40da-a912-83cc536e502e None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:55.516112 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598e0dcc-aeda-40da-a912-83cc536e502e None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:55.517457 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 150/322] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:25:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 22:25:55.522270 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918caf74-987e-4a63-b255-d4e4b80cdd5c None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:55.522562 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918caf74-987e-4a63-b255-d4e4b80cdd5c None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:55.522868 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918caf74-987e-4a63-b255-d4e4b80cdd5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:55.539495 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-918caf74-987e-4a63-b255-d4e4b80cdd5c None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:25:55.540688 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-918caf74-987e-4a63-b255-d4e4b80cdd5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:25:55.545875 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 173/323] 10.222.0.81 () {60 vars in 1134 bytes} [Wed Jul 20 22:25:55 2022] POST /identity/v3/groups => generated 226 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 22:25:55.621429 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 151/324] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:25:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 22:25:55.644740 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c18000-0125-455b-887a-884f929bdbdc None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:55.647752 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:55.753952 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c18000-0125-455b-887a-884f929bdbdc None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:55.792300 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746dbf69-275c-4bff-aa6c-3ceee5cfa014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:55.793686 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746dbf69-275c-4bff-aa6c-3ceee5cfa014 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:55.794143 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746dbf69-275c-4bff-aa6c-3ceee5cfa014 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:55.794607 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746dbf69-275c-4bff-aa6c-3ceee5cfa014 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:55.807413 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746dbf69-275c-4bff-aa6c-3ceee5cfa014 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 22:25:55.816055 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-746dbf69-275c-4bff-aa6c-3ceee5cfa014 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 22:25:55.816055 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:25:55.816055 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:25:55.816055 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:25:55.816055 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:25:55.816055 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:25:55.816055 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:25:55.816055 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:25:55.816055 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:25:55.816055 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:25:55.816055 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:25:55.816055 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:25:55.816055 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 22:25:55.816055 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 22:25:55.816055 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 22:25:55.816055 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 22:25:55.817458 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 22:25:55.817458 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 22:25:55.817458 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 22:25:55.817458 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 22:25:55.817458 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:25:55.817458 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:25:55.817458 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:25:55.817458 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:25:55.817458 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:25:55.817458 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:25:55.817458 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:25:55.817458 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:25:55.817458 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:25:55.817458 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:25:55.817458 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:25:55.817458 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:25:55.818275 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:25:55.818275 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:25:55.818275 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:25:55.818275 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:25:55.818275 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 22:25:55.818275 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 22:25:55.818275 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 22:25:55.818275 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 22:25:55.818275 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 22:25:55.818275 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 22:25:55.818275 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 22:25:55.818275 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:25:55.818275 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 174/325] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Jul 20 22:25:55 2022] GET /identity/v3/roles/reader => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:25:55.823375 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918caf74-987e-4a63-b255-d4e4b80cdd5c None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:55.876685 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:55.932801 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918caf74-987e-4a63-b255-d4e4b80cdd5c None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:55.966996 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6db759-bbd7-4d74-a83d-3375f7b777f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:55.967710 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6db759-bbd7-4d74-a83d-3375f7b777f9 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:55.967997 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6db759-bbd7-4d74-a83d-3375f7b777f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:55.968279 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6db759-bbd7-4d74-a83d-3375f7b777f9 None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:55.969035 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6db759-bbd7-4d74-a83d-3375f7b777f9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:25:55.969344 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6db759-bbd7-4d74-a83d-3375f7b777f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:55.970727 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6db759-bbd7-4d74-a83d-3375f7b777f9 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:55.980064 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 152/326] 10.222.0.81 () {56 vars in 1101 bytes} [Wed Jul 20 22:25:55 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:25:55.985582 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746dbf69-275c-4bff-aa6c-3ceee5cfa014 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:56.051435 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:56.094409 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746dbf69-275c-4bff-aa6c-3ceee5cfa014 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:56.131469 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae41421-505a-43bf-ba79-826d9737c4d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:56.132293 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae41421-505a-43bf-ba79-826d9737c4d1 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:56.132578 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae41421-505a-43bf-ba79-826d9737c4d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:56.132871 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae41421-505a-43bf-ba79-826d9737c4d1 None admin] PATH_INFO: `/v3/users/9bf09b5a20154dc7a8a0645b0f9a9d97` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:56.134167 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae41421-505a-43bf-ba79-826d9737c4d1 None admin] RBAC: Authorizing `identity:get_user(user_id=9bf09b5a20154dc7a8a0645b0f9a9d97)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:56.135681 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae41421-505a-43bf-ba79-826d9737c4d1 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:56.142116 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 175/327] 10.222.0.81 () {56 vars in 1177 bytes} [Wed Jul 20 22:25:55 2022] GET /identity/v3/users/9bf09b5a20154dc7a8a0645b0f9a9d97 => generated 293 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:25:56.143905 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6db759-bbd7-4d74-a83d-3375f7b777f9 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:56.189522 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05711f5d-5e92-4a24-b032-95fe4433497b None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:56.189868 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05711f5d-5e92-4a24-b032-95fe4433497b None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:56.190094 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05711f5d-5e92-4a24-b032-95fe4433497b None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:56.191807 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 176/328] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:25:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:25:56.197557 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1766814f-b2d5-44bd-a51a-c6619439dba0 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:56.197859 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1766814f-b2d5-44bd-a51a-c6619439dba0 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:56.198146 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1766814f-b2d5-44bd-a51a-c6619439dba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:56.219121 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-1766814f-b2d5-44bd-a51a-c6619439dba0 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:25:56.220602 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-1766814f-b2d5-44bd-a51a-c6619439dba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:25:56.267416 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6db759-bbd7-4d74-a83d-3375f7b777f9 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:56.282179 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:56.315256 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 177/329] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:25:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 22:25:56.326694 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b02e8d8-b7ef-4c1a-b954-ae06de0b049f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:56.327641 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b02e8d8-b7ef-4c1a-b954-ae06de0b049f None admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:56.327933 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b02e8d8-b7ef-4c1a-b954-ae06de0b049f None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:56.328219 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b02e8d8-b7ef-4c1a-b954-ae06de0b049f None admin] PATH_INFO: `/v3/system/users/9bf09b5a20154dc7a8a0645b0f9a9d97/roles/5b2ef0feadf94c059dffbd7d8bc03aa4` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:56.329741 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b02e8d8-b7ef-4c1a-b954-ae06de0b049f None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=9bf09b5a20154dc7a8a0645b0f9a9d97, role_id=5b2ef0feadf94c059dffbd7d8bc03aa4)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:56.331165 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b02e8d8-b7ef-4c1a-b954-ae06de0b049f None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:56.347546 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1766814f-b2d5-44bd-a51a-c6619439dba0 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:56.386282 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 153/330] 10.222.0.81 () {58 vars in 1334 bytes} [Wed Jul 20 22:25:56 2022] PUT /identity/v3/system/users/9bf09b5a20154dc7a8a0645b0f9a9d97/roles/5b2ef0feadf94c059dffbd7d8bc03aa4 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:25:56.420123 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:56.467716 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1766814f-b2d5-44bd-a51a-c6619439dba0 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:56.506708 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e2e571-20ae-49cd-bc5e-0191b579f8f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:56.507588 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e2e571-20ae-49cd-bc5e-0191b579f8f0 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:56.507893 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e2e571-20ae-49cd-bc5e-0191b579f8f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:56.508182 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e2e571-20ae-49cd-bc5e-0191b579f8f0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:56.521663 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e2e571-20ae-49cd-bc5e-0191b579f8f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 22:25:56.530308 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-90e2e571-20ae-49cd-bc5e-0191b579f8f0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 22:25:56.530308 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:25:56.530308 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:25:56.530308 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:25:56.530308 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:25:56.530308 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:25:56.530308 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:25:56.530308 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:25:56.530308 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:25:56.530308 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:25:56.530308 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:25:56.530308 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:25:56.530308 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 22:25:56.530308 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 22:25:56.530308 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 22:25:56.530308 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 22:25:56.531338 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 22:25:56.531338 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 22:25:56.531338 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 22:25:56.531338 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 22:25:56.531338 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:25:56.531338 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:25:56.531338 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:25:56.531338 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:25:56.531338 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:25:56.531338 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:25:56.531338 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:25:56.531338 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:25:56.531338 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:25:56.531338 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:25:56.531338 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:25:56.531338 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:25:56.532144 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:25:56.532144 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:25:56.532144 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:25:56.532144 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:25:56.532144 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 22:25:56.532144 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 22:25:56.532144 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 22:25:56.532144 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 22:25:56.532144 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 22:25:56.532144 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 22:25:56.532144 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 22:25:56.532144 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:25:56.532144 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 178/331] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Jul 20 22:25:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:25:56.540061 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b02e8d8-b7ef-4c1a-b954-ae06de0b049f None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:56.651235 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b02e8d8-b7ef-4c1a-b954-ae06de0b049f None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:56.689634 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb44c33f-2da1-435e-b1f1-e2aa67d92a0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:56.690317 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb44c33f-2da1-435e-b1f1-e2aa67d92a0d None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:56.690604 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb44c33f-2da1-435e-b1f1-e2aa67d92a0d None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:56.690890 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb44c33f-2da1-435e-b1f1-e2aa67d92a0d None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:56.691673 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb44c33f-2da1-435e-b1f1-e2aa67d92a0d None admin] RBAC: Adding query filter params (name=member) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:25:56.691984 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb44c33f-2da1-435e-b1f1-e2aa67d92a0d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:56.693349 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb44c33f-2da1-435e-b1f1-e2aa67d92a0d None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:56.702400 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 154/332] 10.222.0.81 () {56 vars in 1101 bytes} [Wed Jul 20 22:25:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:25:56.707788 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e2e571-20ae-49cd-bc5e-0191b579f8f0 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:56.818963 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e2e571-20ae-49cd-bc5e-0191b579f8f0 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:56.841007 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:56.856350 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587bf93e-8f68-4628-a8de-7624a59a0451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:56.857308 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587bf93e-8f68-4628-a8de-7624a59a0451 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:56.857644 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587bf93e-8f68-4628-a8de-7624a59a0451 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:56.857903 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587bf93e-8f68-4628-a8de-7624a59a0451 None admin] PATH_INFO: `/v3/users/f3e955b84b7d4c9d91b15e961e4ec65e` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:56.859215 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587bf93e-8f68-4628-a8de-7624a59a0451 None admin] RBAC: Authorizing `identity:get_user(user_id=f3e955b84b7d4c9d91b15e961e4ec65e)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:56.860688 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587bf93e-8f68-4628-a8de-7624a59a0451 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:56.862679 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 179/333] 10.222.0.81 () {56 vars in 1177 bytes} [Wed Jul 20 22:25:56 2022] GET /identity/v3/users/f3e955b84b7d4c9d91b15e961e4ec65e => generated 293 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 22:25:56.868398 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb44c33f-2da1-435e-b1f1-e2aa67d92a0d None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:56.921873 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:57.026077 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb44c33f-2da1-435e-b1f1-e2aa67d92a0d None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:57.063708 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21398879-a70c-454b-90cd-51a51de1055c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:57.064378 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21398879-a70c-454b-90cd-51a51de1055c None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:57.064664 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21398879-a70c-454b-90cd-51a51de1055c None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:57.064941 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21398879-a70c-454b-90cd-51a51de1055c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:57.065777 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21398879-a70c-454b-90cd-51a51de1055c None admin] RBAC: Adding query filter params (role.id=559942c4de1744169f8b5e0404ecde6b, user.id=f3e955b84b7d4c9d91b15e961e4ec65e, scope.system=all) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:25:57.066096 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21398879-a70c-454b-90cd-51a51de1055c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:57.067562 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21398879-a70c-454b-90cd-51a51de1055c None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:57.087377 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 155/334] 10.222.0.81 () {56 vars in 1308 bytes} [Wed Jul 20 22:25:56 2022] GET /identity/v3/role_assignments?user.id=f3e955b84b7d4c9d91b15e961e4ec65e&scope.system=all&role.id=559942c4de1744169f8b5e0404ecde6b => generated 510 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:25:58.471808 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec92e15-f300-4937-8453-967c16da6e5d None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:58.472849 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec92e15-f300-4937-8453-967c16da6e5d None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:58.473365 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec92e15-f300-4937-8453-967c16da6e5d None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:58.475206 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 180/335] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:25:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:25:58.481814 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fb5dee-9d73-44fc-a8a1-650a4e03ef0b None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:58.482652 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fb5dee-9d73-44fc-a8a1-650a4e03ef0b None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:58.483177 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fb5dee-9d73-44fc-a8a1-650a4e03ef0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:58.501928 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-13fb5dee-9d73-44fc-a8a1-650a4e03ef0b None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:25:58.503522 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-13fb5dee-9d73-44fc-a8a1-650a4e03ef0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:25:58.578317 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 156/336] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:25:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 22:25:58.602729 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec92e15-f300-4937-8453-967c16da6e5d None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:58.738662 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec92e15-f300-4937-8453-967c16da6e5d None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:58.781472 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b296d2b-ef1f-4563-a3a7-53a571646268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:58.782657 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b296d2b-ef1f-4563-a3a7-53a571646268 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:58.783186 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b296d2b-ef1f-4563-a3a7-53a571646268 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:58.783668 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b296d2b-ef1f-4563-a3a7-53a571646268 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:58.803899 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b296d2b-ef1f-4563-a3a7-53a571646268 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 22:25:58.807926 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99559084-92dd-4227-b407-34c10a5c5d02 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:58.808224 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99559084-92dd-4227-b407-34c10a5c5d02 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:58.808509 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99559084-92dd-4227-b407-34c10a5c5d02 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:58.810104 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 157/337] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:25:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 22:25:58.813163 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-4b296d2b-ef1f-4563-a3a7-53a571646268 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 22:25:58.813163 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:25:58.813163 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:25:58.813163 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:25:58.813163 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:25:58.813163 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:25:58.813163 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:25:58.813163 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:25:58.813163 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:25:58.813163 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:25:58.813163 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:25:58.813163 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:25:58.813163 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 22:25:58.813163 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 22:25:58.813163 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 22:25:58.813163 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 22:25:58.814034 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 22:25:58.814034 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 22:25:58.814034 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 22:25:58.814034 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 22:25:58.814034 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:25:58.814034 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:25:58.814034 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:25:58.814034 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:25:58.814034 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:25:58.814034 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:25:58.814034 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:25:58.814034 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:25:58.814034 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:25:58.814034 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:25:58.814034 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:25:58.814034 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:25:58.814811 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:25:58.814811 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:25:58.814811 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:25:58.814811 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:25:58.814811 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 22:25:58.814811 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 22:25:58.814811 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 22:25:58.814811 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 22:25:58.814811 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 22:25:58.814811 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 22:25:58.814811 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 22:25:58.814811 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:25:58.814811 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 181/338] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Jul 20 22:25:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:25:58.820843 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99559084-92dd-4227-b407-34c10a5c5d02 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:58.823514 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade25f7f-64a1-48ba-b261-abeb2b66d0fa None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:58.823810 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade25f7f-64a1-48ba-b261-abeb2b66d0fa None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:58.827108 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade25f7f-64a1-48ba-b261-abeb2b66d0fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:58.848820 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-ade25f7f-64a1-48ba-b261-abeb2b66d0fa None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:25:58.850311 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-ade25f7f-64a1-48ba-b261-abeb2b66d0fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:25:58.872805 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:58.929392 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:58.987204 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99559084-92dd-4227-b407-34c10a5c5d02 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:58.989643 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 182/339] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:25:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 22:25:59.026244 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade25f7f-64a1-48ba-b261-abeb2b66d0fa None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:59.048092 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd265d9c-19f9-440e-8a96-af3a14099aff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:59.049079 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd265d9c-19f9-440e-8a96-af3a14099aff None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:59.049462 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd265d9c-19f9-440e-8a96-af3a14099aff None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:59.049838 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd265d9c-19f9-440e-8a96-af3a14099aff None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:59.051079 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd265d9c-19f9-440e-8a96-af3a14099aff None admin] RBAC: Adding query filter params (name=member) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:25:59.051518 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd265d9c-19f9-440e-8a96-af3a14099aff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:59.053589 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd265d9c-19f9-440e-8a96-af3a14099aff None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:59.070025 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 158/340] 10.222.0.81 () {56 vars in 1101 bytes} [Wed Jul 20 22:25:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:25:59.075935 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd265d9c-19f9-440e-8a96-af3a14099aff None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:59.217372 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade25f7f-64a1-48ba-b261-abeb2b66d0fa None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:59.269142 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:59.277100 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206c0b0d-747d-448a-8dff-5ac58658188d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:59.286884 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206c0b0d-747d-448a-8dff-5ac58658188d None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:59.286884 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206c0b0d-747d-448a-8dff-5ac58658188d None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:59.286884 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206c0b0d-747d-448a-8dff-5ac58658188d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:59.291141 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206c0b0d-747d-448a-8dff-5ac58658188d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 22:25:59.303662 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-206c0b0d-747d-448a-8dff-5ac58658188d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 22:25:59.303662 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:25:59.303662 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:25:59.303662 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:25:59.303662 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:25:59.303662 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:25:59.303662 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:25:59.303662 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:25:59.303662 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:25:59.303662 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:25:59.303662 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:25:59.303662 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:25:59.303662 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 22:25:59.303662 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 22:25:59.303662 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 22:25:59.303662 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 22:25:59.304718 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 22:25:59.304718 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 22:25:59.304718 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 22:25:59.304718 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 22:25:59.304718 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:25:59.304718 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:25:59.304718 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:25:59.304718 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:25:59.304718 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:25:59.304718 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:25:59.304718 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:25:59.304718 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:25:59.304718 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:25:59.304718 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:25:59.304718 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:25:59.304718 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:25:59.305528 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:25:59.305528 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:25:59.305528 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:25:59.305528 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:25:59.305528 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 22:25:59.305528 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 22:25:59.305528 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 22:25:59.305528 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 22:25:59.305528 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 22:25:59.305528 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 22:25:59.305528 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 22:25:59.305528 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:25:59.306160 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 183/341] 10.222.0.81 () {56 vars in 1114 bytes} [Wed Jul 20 22:25:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 22:25:59.311347 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206c0b0d-747d-448a-8dff-5ac58658188d None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:59.312156 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd265d9c-19f9-440e-8a96-af3a14099aff None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:59.360809 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:59.380322 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becc38b1-b5bf-48cd-b65d-863b43fdcd07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:59.381331 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becc38b1-b5bf-48cd-b65d-863b43fdcd07 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:59.381856 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becc38b1-b5bf-48cd-b65d-863b43fdcd07 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:59.382343 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becc38b1-b5bf-48cd-b65d-863b43fdcd07 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:59.437131 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206c0b0d-747d-448a-8dff-5ac58658188d None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:59.472012 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becc38b1-b5bf-48cd-b65d-863b43fdcd07 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:59.473517 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becc38b1-b5bf-48cd-b65d-863b43fdcd07 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:59.492659 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bce25ab-334d-4ab1-9efe-5ccdd19915fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:59.493341 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bce25ab-334d-4ab1-9efe-5ccdd19915fe None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:59.493633 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bce25ab-334d-4ab1-9efe-5ccdd19915fe None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:59.493917 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bce25ab-334d-4ab1-9efe-5ccdd19915fe None admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:59.494681 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bce25ab-334d-4ab1-9efe-5ccdd19915fe None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:25:59.494995 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bce25ab-334d-4ab1-9efe-5ccdd19915fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:59.496438 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bce25ab-334d-4ab1-9efe-5ccdd19915fe None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:59.510128 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 184/342] 10.222.0.81 () {56 vars in 1111 bytes} [Wed Jul 20 22:25:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 22:25:59.528729 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bce25ab-334d-4ab1-9efe-5ccdd19915fe None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:59.570741 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:59.595887 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 159/343] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Jul 20 22:25:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 521 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 22:25:59.597595 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68ba0a8-7bab-4164-a832-437b685df175 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:59.597889 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68ba0a8-7bab-4164-a832-437b685df175 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:59.598167 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68ba0a8-7bab-4164-a832-437b685df175 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:59.599726 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 160/344] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:25:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:25:59.602634 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68ba0a8-7bab-4164-a832-437b685df175 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:25:59.687537 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:59.692367 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bce25ab-334d-4ab1-9efe-5ccdd19915fe None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:59.727047 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68ba0a8-7bab-4164-a832-437b685df175 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:25:59.743525 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2638a60-aa2a-44e3-8c94-b9044dd5b1fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:59.744474 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2638a60-aa2a-44e3-8c94-b9044dd5b1fe None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:59.744967 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2638a60-aa2a-44e3-8c94-b9044dd5b1fe None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:59.745464 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2638a60-aa2a-44e3-8c94-b9044dd5b1fe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:59.777820 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84adae5-0655-47c5-9856-0c7b302094d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:25:59.780036 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84adae5-0655-47c5-9856-0c7b302094d5 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:59.780483 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84adae5-0655-47c5-9856-0c7b302094d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:59.780920 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84adae5-0655-47c5-9856-0c7b302094d5 None admin] PATH_INFO: `/v3/projects/da54f6860e1f43c29cf90271280bdcae` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:59.782781 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84adae5-0655-47c5-9856-0c7b302094d5 None admin] RBAC: Authorizing `identity:get_project(project_id=da54f6860e1f43c29cf90271280bdcae)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:59.786555 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84adae5-0655-47c5-9856-0c7b302094d5 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:59.789872 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 161/345] 10.222.0.81 () {56 vars in 1186 bytes} [Wed Jul 20 22:25:59 2022] GET /identity/v3/projects/da54f6860e1f43c29cf90271280bdcae => generated 298 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:25:59.792939 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572927a6-f000-4e5f-a8f0-826b54b0d3c2 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:25:59.794170 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572927a6-f000-4e5f-a8f0-826b54b0d3c2 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:25:59.794606 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572927a6-f000-4e5f-a8f0-826b54b0d3c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:25:59.828496 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-572927a6-f000-4e5f-a8f0-826b54b0d3c2 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:25:59.830117 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-572927a6-f000-4e5f-a8f0-826b54b0d3c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:25:59.872541 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2638a60-aa2a-44e3-8c94-b9044dd5b1fe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:25:59.874269 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2638a60-aa2a-44e3-8c94-b9044dd5b1fe None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:25:59.905978 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:25:59.932238 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 162/346] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:25:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 22:25:59.933451 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572927a6-f000-4e5f-a8f0-826b54b0d3c2 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:00.000410 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:00.005707 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 185/347] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Jul 20 22:25:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 477 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 22:26:00.010344 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c24b1e-3d11-4110-abf6-741d7c6f7e5b None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:00.010344 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c24b1e-3d11-4110-abf6-741d7c6f7e5b None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:00.010344 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c24b1e-3d11-4110-abf6-741d7c6f7e5b None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:00.011409 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 186/348] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:26:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:26:00.015574 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c24b1e-3d11-4110-abf6-741d7c6f7e5b None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:00.073790 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572927a6-f000-4e5f-a8f0-826b54b0d3c2 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:00.115179 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6593785-11d2-437e-a1ae-5a7ef26decb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:00.116477 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6593785-11d2-437e-a1ae-5a7ef26decb7 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:00.117759 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6593785-11d2-437e-a1ae-5a7ef26decb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:00.118190 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6593785-11d2-437e-a1ae-5a7ef26decb7 None admin] PATH_INFO: `/v3/groups/f5b13eb0527d4f748d667e2b1d087e54` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:00.127710 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6593785-11d2-437e-a1ae-5a7ef26decb7 None admin] RBAC: Authorizing `identity:get_group(group_id=f5b13eb0527d4f748d667e2b1d087e54)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:00.129450 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6593785-11d2-437e-a1ae-5a7ef26decb7 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:00.131550 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 163/349] 10.222.0.81 () {56 vars in 1180 bytes} [Wed Jul 20 22:25:59 2022] GET /identity/v3/groups/f5b13eb0527d4f748d667e2b1d087e54 => generated 226 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:00.133576 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2accf65b-4b51-4243-99d8-069596ff0040 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:00.134131 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2accf65b-4b51-4243-99d8-069596ff0040 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:00.134577 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2accf65b-4b51-4243-99d8-069596ff0040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:00.135236 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c24b1e-3d11-4110-abf6-741d7c6f7e5b None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:00.155794 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-2accf65b-4b51-4243-99d8-069596ff0040 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:26:00.157419 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-2accf65b-4b51-4243-99d8-069596ff0040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:26:00.178688 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b009da-43f6-4aea-b467-5953e58c4b94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:00.180236 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b009da-43f6-4aea-b467-5953e58c4b94 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:00.180531 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b009da-43f6-4aea-b467-5953e58c4b94 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:00.180813 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b009da-43f6-4aea-b467-5953e58c4b94 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:00.194611 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b009da-43f6-4aea-b467-5953e58c4b94 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 22:26:00.206453 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-53b009da-43f6-4aea-b467-5953e58c4b94 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 22:26:00.206453 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:00.206453 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:00.206453 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:00.206453 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:00.206453 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:00.206453 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:00.206453 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:00.206453 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:00.206453 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:00.206453 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:00.206453 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:00.206453 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 22:26:00.206453 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 22:26:00.206453 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 22:26:00.206453 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 22:26:00.207431 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 22:26:00.207431 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 22:26:00.207431 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 22:26:00.207431 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 22:26:00.207431 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:00.207431 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:00.207431 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:00.207431 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:00.207431 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:00.207431 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:00.207431 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:00.207431 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:00.207431 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:00.207431 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:00.207431 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:00.207431 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:00.208240 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:00.208240 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:00.208240 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:00.208240 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:00.208240 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 22:26:00.208240 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 22:26:00.208240 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 22:26:00.208240 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 22:26:00.208240 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 22:26:00.208240 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 22:26:00.208240 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 22:26:00.208240 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:26:00.208240 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 187/350] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Jul 20 22:26:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:00.208946 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b009da-43f6-4aea-b467-5953e58c4b94 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:00.223118 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:00.259127 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 164/351] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:26:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 22:26:00.260312 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2accf65b-4b51-4243-99d8-069596ff0040 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:00.303363 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:00.334043 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b009da-43f6-4aea-b467-5953e58c4b94 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:00.379498 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50edc57f-ca84-4aa6-a69a-1020e9b4f71d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:00.380732 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50edc57f-ca84-4aa6-a69a-1020e9b4f71d None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:00.380856 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2accf65b-4b51-4243-99d8-069596ff0040 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:00.381337 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50edc57f-ca84-4aa6-a69a-1020e9b4f71d None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:00.381837 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50edc57f-ca84-4aa6-a69a-1020e9b4f71d None admin] PATH_INFO: `/v3/projects/da54f6860e1f43c29cf90271280bdcae` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:00.384085 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50edc57f-ca84-4aa6-a69a-1020e9b4f71d None admin] RBAC: Authorizing `identity:get_project(project_id=da54f6860e1f43c29cf90271280bdcae)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:00.386333 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50edc57f-ca84-4aa6-a69a-1020e9b4f71d None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:00.387510 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 188/352] 10.222.0.81 () {56 vars in 1186 bytes} [Wed Jul 20 22:26:00 2022] GET /identity/v3/projects/da54f6860e1f43c29cf90271280bdcae => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:00.388305 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50edc57f-ca84-4aa6-a69a-1020e9b4f71d None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:00.432057 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc7fd40-56e7-4570-bb3f-b9b93e8a32b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:00.432735 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc7fd40-56e7-4570-bb3f-b9b93e8a32b2 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:00.433024 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc7fd40-56e7-4570-bb3f-b9b93e8a32b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:00.433301 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc7fd40-56e7-4570-bb3f-b9b93e8a32b2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:00.434141 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc7fd40-56e7-4570-bb3f-b9b93e8a32b2 None admin] RBAC: Adding query filter params (group.id=f5b13eb0527d4f748d667e2b1d087e54, role.id=559942c4de1744169f8b5e0404ecde6b, scope.project.id=da54f6860e1f43c29cf90271280bdcae) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:00.434458 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc7fd40-56e7-4570-bb3f-b9b93e8a32b2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:00.435955 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc7fd40-56e7-4570-bb3f-b9b93e8a32b2 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:00.442520 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 165/353] 10.222.0.81 () {56 vars in 1376 bytes} [Wed Jul 20 22:26:00 2022] GET /identity/v3/role_assignments?group.id=f5b13eb0527d4f748d667e2b1d087e54&scope.project.id=da54f6860e1f43c29cf90271280bdcae&role.id=559942c4de1744169f8b5e0404ecde6b => generated 260 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:00.444115 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dde6ac-ca5a-47b6-bb44-3698dce26704 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:00.444477 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dde6ac-ca5a-47b6-bb44-3698dce26704 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:00.444834 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dde6ac-ca5a-47b6-bb44-3698dce26704 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:00.446835 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 166/354] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:26:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:26:00.450181 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dde6ac-ca5a-47b6-bb44-3698dce26704 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:00.509158 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:00.518817 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50edc57f-ca84-4aa6-a69a-1020e9b4f71d None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:00.567569 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:00.582640 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19604399-31bd-451d-8d53-322727bbc8a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:00.583549 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19604399-31bd-451d-8d53-322727bbc8a8 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:00.583838 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19604399-31bd-451d-8d53-322727bbc8a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:00.584189 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19604399-31bd-451d-8d53-322727bbc8a8 None admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:00.584906 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19604399-31bd-451d-8d53-322727bbc8a8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:00.585225 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19604399-31bd-451d-8d53-322727bbc8a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:00.586704 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19604399-31bd-451d-8d53-322727bbc8a8 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:00.592694 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dde6ac-ca5a-47b6-bb44-3698dce26704 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:00.597744 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 189/355] 10.222.0.81 () {56 vars in 1101 bytes} [Wed Jul 20 22:26:00 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:00.598447 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19604399-31bd-451d-8d53-322727bbc8a8 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:00.640411 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cef259-3c60-45f3-a1de-19be89beacd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:00.641377 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cef259-3c60-45f3-a1de-19be89beacd1 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:00.641738 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cef259-3c60-45f3-a1de-19be89beacd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:00.642083 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cef259-3c60-45f3-a1de-19be89beacd1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:00.655510 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cef259-3c60-45f3-a1de-19be89beacd1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 22:26:00.664652 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-96cef259-3c60-45f3-a1de-19be89beacd1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 22:26:00.664652 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:00.664652 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:00.664652 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:00.664652 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:00.664652 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:00.664652 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:00.664652 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:00.664652 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:00.664652 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:00.664652 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:00.664652 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:00.664652 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 22:26:00.664652 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 22:26:00.664652 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 22:26:00.664652 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 22:26:00.665597 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 22:26:00.665597 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 22:26:00.665597 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 22:26:00.665597 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 22:26:00.665597 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:00.665597 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:00.665597 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:00.665597 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:00.665597 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:00.665597 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:00.665597 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:00.665597 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:00.665597 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:00.665597 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:00.665597 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:00.665597 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:00.666406 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:00.666406 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:00.666406 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:00.666406 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:00.666406 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 22:26:00.666406 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 22:26:00.666406 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 22:26:00.666406 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 22:26:00.666406 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 22:26:00.666406 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 22:26:00.666406 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 22:26:00.666406 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:26:00.666406 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 167/356] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Jul 20 22:26:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:00.667808 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c76f061-5547-4510-8bb0-85258435f354 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:00.667916 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c76f061-5547-4510-8bb0-85258435f354 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:00.668404 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c76f061-5547-4510-8bb0-85258435f354 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:00.669548 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 168/357] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:26:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:26:00.670459 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd60369-0bb7-4270-9f0e-06def3d6d815 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:00.670746 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd60369-0bb7-4270-9f0e-06def3d6d815 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:00.671085 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd60369-0bb7-4270-9f0e-06def3d6d815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:00.694577 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-8bd60369-0bb7-4270-9f0e-06def3d6d815 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:26:00.696395 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-8bd60369-0bb7-4270-9f0e-06def3d6d815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:26:00.735763 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19604399-31bd-451d-8d53-322727bbc8a8 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:00.779501 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921b26c3-a987-423e-b55e-ec00b17412fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:00.780504 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921b26c3-a987-423e-b55e-ec00b17412fd None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:00.780793 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921b26c3-a987-423e-b55e-ec00b17412fd None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:00.781073 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921b26c3-a987-423e-b55e-ec00b17412fd None admin] PATH_INFO: `/v3/groups/f5b13eb0527d4f748d667e2b1d087e54` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:00.782457 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 169/358] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:26:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 22:26:00.785239 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd60369-0bb7-4270-9f0e-06def3d6d815 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:00.785358 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921b26c3-a987-423e-b55e-ec00b17412fd None admin] RBAC: Authorizing `identity:get_group(group_id=f5b13eb0527d4f748d667e2b1d087e54)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:00.785728 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921b26c3-a987-423e-b55e-ec00b17412fd None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:00.788076 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 190/359] 10.222.0.81 () {56 vars in 1180 bytes} [Wed Jul 20 22:26:00 2022] GET /identity/v3/groups/f5b13eb0527d4f748d667e2b1d087e54 => generated 226 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:00.788490 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921b26c3-a987-423e-b55e-ec00b17412fd None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:00.820467 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:00.860791 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:00.912491 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd60369-0bb7-4270-9f0e-06def3d6d815 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:00.920951 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921b26c3-a987-423e-b55e-ec00b17412fd None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:00.953631 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c36dd1-23d4-479a-8df1-b2052185ab87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:00.954533 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c36dd1-23d4-479a-8df1-b2052185ab87 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:00.954820 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c36dd1-23d4-479a-8df1-b2052185ab87 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:00.955119 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c36dd1-23d4-479a-8df1-b2052185ab87 None admin] PATH_INFO: `/v3/users/9bf09b5a20154dc7a8a0645b0f9a9d97` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:00.956350 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c36dd1-23d4-479a-8df1-b2052185ab87 None admin] RBAC: Authorizing `identity:get_user(user_id=9bf09b5a20154dc7a8a0645b0f9a9d97)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:00.957799 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c36dd1-23d4-479a-8df1-b2052185ab87 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:00.959501 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 170/360] 10.222.0.81 () {56 vars in 1177 bytes} [Wed Jul 20 22:26:00 2022] GET /identity/v3/users/9bf09b5a20154dc7a8a0645b0f9a9d97 => generated 293 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:00.960733 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1b5cca-93ee-48a7-9483-96b53706f73e None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:00.961023 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1b5cca-93ee-48a7-9483-96b53706f73e None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:00.961805 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1b5cca-93ee-48a7-9483-96b53706f73e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:00.963638 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0eaf354-eac7-4f90-b63e-b73e29b7acf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:00.964321 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eaf354-eac7-4f90-b63e-b73e29b7acf2 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:00.964609 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eaf354-eac7-4f90-b63e-b73e29b7acf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:00.964891 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0eaf354-eac7-4f90-b63e-b73e29b7acf2 None admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:00.965653 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0eaf354-eac7-4f90-b63e-b73e29b7acf2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:00.965968 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0eaf354-eac7-4f90-b63e-b73e29b7acf2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:00.967386 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0eaf354-eac7-4f90-b63e-b73e29b7acf2 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:00.977379 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 191/361] 10.222.0.81 () {56 vars in 1101 bytes} [Wed Jul 20 22:26:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:00.979975 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0eaf354-eac7-4f90-b63e-b73e29b7acf2 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:00.984707 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-ee1b5cca-93ee-48a7-9483-96b53706f73e None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:26:00.985898 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-ee1b5cca-93ee-48a7-9483-96b53706f73e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:26:01.067625 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 171/362] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:26:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 22:26:01.069404 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1b5cca-93ee-48a7-9483-96b53706f73e None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:01.108042 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:01.125283 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0eaf354-eac7-4f90-b63e-b73e29b7acf2 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:01.151680 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:01.169567 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf534dc6-83f6-490c-9ac0-dfb7f8cc0415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:01.170361 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf534dc6-83f6-490c-9ac0-dfb7f8cc0415 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:01.170651 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf534dc6-83f6-490c-9ac0-dfb7f8cc0415 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:01.170937 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf534dc6-83f6-490c-9ac0-dfb7f8cc0415 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:01.171834 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf534dc6-83f6-490c-9ac0-dfb7f8cc0415 None admin] RBAC: Adding query filter params (group.id=f5b13eb0527d4f748d667e2b1d087e54, role.id=6d5a3e6a22fa4f4fa47afcfce0246a1f, scope.project.id=da54f6860e1f43c29cf90271280bdcae) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:01.172202 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf534dc6-83f6-490c-9ac0-dfb7f8cc0415 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:01.174384 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf534dc6-83f6-490c-9ac0-dfb7f8cc0415 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:01.181220 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 192/363] 10.222.0.81 () {56 vars in 1376 bytes} [Wed Jul 20 22:26:00 2022] GET /identity/v3/role_assignments?group.id=f5b13eb0527d4f748d667e2b1d087e54&scope.project.id=da54f6860e1f43c29cf90271280bdcae&role.id=6d5a3e6a22fa4f4fa47afcfce0246a1f => generated 260 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:01.182063 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf534dc6-83f6-490c-9ac0-dfb7f8cc0415 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:01.207050 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1b5cca-93ee-48a7-9483-96b53706f73e None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:01.247570 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6805bbc9-735d-4df7-bd12-28976f36fb28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:01.248747 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6805bbc9-735d-4df7-bd12-28976f36fb28 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:01.249451 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6805bbc9-735d-4df7-bd12-28976f36fb28 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:01.249734 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6805bbc9-735d-4df7-bd12-28976f36fb28 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:01.266399 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6805bbc9-735d-4df7-bd12-28976f36fb28 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 22:26:01.277573 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-6805bbc9-735d-4df7-bd12-28976f36fb28 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 22:26:01.277573 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:01.277573 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:01.277573 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:01.277573 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:01.277573 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:01.277573 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:01.277573 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:01.277573 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:01.277573 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:01.277573 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:01.277573 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:01.277573 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 22:26:01.277573 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 22:26:01.277573 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 22:26:01.277573 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 22:26:01.278541 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 22:26:01.278541 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 22:26:01.278541 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 22:26:01.278541 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 22:26:01.278541 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:01.278541 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:01.278541 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:01.278541 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:01.278541 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:01.278541 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:01.278541 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:01.278541 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:01.278541 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:01.278541 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:01.278541 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:01.278541 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:01.279403 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:01.279403 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:01.279403 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:01.279403 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:01.279403 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 22:26:01.279403 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 22:26:01.279403 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 22:26:01.279403 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 22:26:01.279403 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 22:26:01.279403 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 22:26:01.279403 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 22:26:01.279403 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:26:01.280825 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 172/364] 10.222.0.81 () {56 vars in 1114 bytes} [Wed Jul 20 22:26:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:01.280825 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6805bbc9-735d-4df7-bd12-28976f36fb28 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:01.308315 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf534dc6-83f6-490c-9ac0-dfb7f8cc0415 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:01.349229 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc96bee2-9691-4fcc-8e8b-19c20ca864b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:01.349904 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc96bee2-9691-4fcc-8e8b-19c20ca864b1 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:01.350190 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc96bee2-9691-4fcc-8e8b-19c20ca864b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:01.350474 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc96bee2-9691-4fcc-8e8b-19c20ca864b1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:01.351454 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc96bee2-9691-4fcc-8e8b-19c20ca864b1 None admin] RBAC: Adding query filter params (role.id=5b2ef0feadf94c059dffbd7d8bc03aa4, user.id=9bf09b5a20154dc7a8a0645b0f9a9d97, scope.system=all) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:01.351778 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc96bee2-9691-4fcc-8e8b-19c20ca864b1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:01.353286 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc96bee2-9691-4fcc-8e8b-19c20ca864b1 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:01.368858 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 193/365] 10.222.0.81 () {56 vars in 1308 bytes} [Wed Jul 20 22:26:01 2022] GET /identity/v3/role_assignments?user.id=9bf09b5a20154dc7a8a0645b0f9a9d97&scope.system=all&role.id=5b2ef0feadf94c059dffbd7d8bc03aa4 => generated 510 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:01.369159 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc96bee2-9691-4fcc-8e8b-19c20ca864b1 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:01.408489 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:01.428924 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:01.434496 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6805bbc9-735d-4df7-bd12-28976f36fb28 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:01.478680 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e8e426-300a-4a1d-9445-d8d9227d19ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:01.479408 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e8e426-300a-4a1d-9445-d8d9227d19ad None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:01.479696 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e8e426-300a-4a1d-9445-d8d9227d19ad None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:01.479976 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e8e426-300a-4a1d-9445-d8d9227d19ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:01.510692 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc96bee2-9691-4fcc-8e8b-19c20ca864b1 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:01.559940 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ae5b85-0fa5-4f99-a8fe-23aae069fc79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:01.560766 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ae5b85-0fa5-4f99-a8fe-23aae069fc79 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:01.561056 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ae5b85-0fa5-4f99-a8fe-23aae069fc79 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:01.561341 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ae5b85-0fa5-4f99-a8fe-23aae069fc79 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:01.567051 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e8e426-300a-4a1d-9445-d8d9227d19ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:01.568893 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e8e426-300a-4a1d-9445-d8d9227d19ad None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:01.575341 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ae5b85-0fa5-4f99-a8fe-23aae069fc79 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 22:26:01.586018 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-96ae5b85-0fa5-4f99-a8fe-23aae069fc79 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 22:26:01.586018 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:01.586018 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:01.586018 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:01.586018 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:01.586018 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:01.586018 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:01.586018 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:01.586018 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:01.586018 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:01.586018 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:01.586018 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:01.586018 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 22:26:01.586018 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 22:26:01.586018 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 22:26:01.586018 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 22:26:01.588118 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 22:26:01.588118 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 22:26:01.588118 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 22:26:01.588118 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 22:26:01.588118 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:01.588118 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:01.588118 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:01.588118 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:01.588118 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:01.588118 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:01.588118 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:01.588118 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:01.588118 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:01.588118 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:01.588118 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:01.588118 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:01.588976 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:01.588976 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:01.588976 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:01.588976 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:01.588976 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 22:26:01.588976 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 22:26:01.588976 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 22:26:01.588976 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 22:26:01.588976 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 22:26:01.588976 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 22:26:01.588976 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 22:26:01.588976 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:26:01.588976 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 194/366] 10.222.0.81 () {56 vars in 1096 bytes} [Wed Jul 20 22:26:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:01.588976 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ae5b85-0fa5-4f99-a8fe-23aae069fc79 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:01.707346 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:01.727500 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 173/367] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Jul 20 22:26:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 22:26:01.728810 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e8e426-300a-4a1d-9445-d8d9227d19ad None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:01.735049 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ae5b85-0fa5-4f99-a8fe-23aae069fc79 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:01.736969 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:01.782709 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7906cd40-f56e-409e-9fa4-263f7ff22184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:01.783428 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7906cd40-f56e-409e-9fa4-263f7ff22184 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:01.783720 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7906cd40-f56e-409e-9fa4-263f7ff22184 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:01.783990 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7906cd40-f56e-409e-9fa4-263f7ff22184 None admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:01.784756 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7906cd40-f56e-409e-9fa4-263f7ff22184 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:01.785065 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7906cd40-f56e-409e-9fa4-263f7ff22184 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:01.786449 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7906cd40-f56e-409e-9fa4-263f7ff22184 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:01.805882 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 195/368] 10.222.0.81 () {56 vars in 1111 bytes} [Wed Jul 20 22:26:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:01.806734 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7906cd40-f56e-409e-9fa4-263f7ff22184 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:01.871648 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e8e426-300a-4a1d-9445-d8d9227d19ad None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:01.920379 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184594c6-8417-4108-932b-b01899c49fda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:01.921469 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184594c6-8417-4108-932b-b01899c49fda None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:01.922330 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184594c6-8417-4108-932b-b01899c49fda None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:01.922813 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184594c6-8417-4108-932b-b01899c49fda None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:01.924544 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184594c6-8417-4108-932b-b01899c49fda None admin] RBAC: Adding query filter params (name=admin) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:01.925046 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184594c6-8417-4108-932b-b01899c49fda None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:01.926637 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184594c6-8417-4108-932b-b01899c49fda None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:01.939481 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 174/369] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Jul 20 22:26:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:01.940581 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184594c6-8417-4108-932b-b01899c49fda None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:01.957329 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7906cd40-f56e-409e-9fa4-263f7ff22184 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:02.001853 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c008d1c-bcab-4764-91e4-9ad652c394f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:02.003266 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c008d1c-bcab-4764-91e4-9ad652c394f9 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:02.003675 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c008d1c-bcab-4764-91e4-9ad652c394f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:02.003965 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c008d1c-bcab-4764-91e4-9ad652c394f9 None admin] PATH_INFO: `/v3/projects/7165d638ee164d27903602ef884fcf03` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:02.006366 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c008d1c-bcab-4764-91e4-9ad652c394f9 None admin] RBAC: Authorizing `identity:get_project(project_id=7165d638ee164d27903602ef884fcf03)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:02.008020 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c008d1c-bcab-4764-91e4-9ad652c394f9 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:02.009018 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:02.010169 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 196/370] 10.222.0.81 () {56 vars in 1186 bytes} [Wed Jul 20 22:26:01 2022] GET /identity/v3/projects/7165d638ee164d27903602ef884fcf03 => generated 302 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:02.011044 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c008d1c-bcab-4764-91e4-9ad652c394f9 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:02.052666 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:02.072827 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184594c6-8417-4108-932b-b01899c49fda None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:02.117155 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea502a6-c977-44dc-a49d-b802397b13d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:02.118063 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea502a6-c977-44dc-a49d-b802397b13d7 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:02.118421 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea502a6-c977-44dc-a49d-b802397b13d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:02.118773 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea502a6-c977-44dc-a49d-b802397b13d7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:02.137573 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c008d1c-bcab-4764-91e4-9ad652c394f9 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:02.180569 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5460b5-6f57-4c80-a84e-e067c3e8c97c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:02.181455 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5460b5-6f57-4c80-a84e-e067c3e8c97c None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:02.182107 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5460b5-6f57-4c80-a84e-e067c3e8c97c None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:02.183409 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5460b5-6f57-4c80-a84e-e067c3e8c97c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:02.211944 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea502a6-c977-44dc-a49d-b802397b13d7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:02.213865 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea502a6-c977-44dc-a49d-b802397b13d7 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:02.263720 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5460b5-6f57-4c80-a84e-e067c3e8c97c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:02.265414 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5460b5-6f57-4c80-a84e-e067c3e8c97c None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:02.282618 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:02.298988 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:02.341666 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 175/371] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Jul 20 22:26:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 22:26:02.342674 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea502a6-c977-44dc-a49d-b802397b13d7 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:02.373905 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 197/372] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Jul 20 22:26:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 22:26:02.374969 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5460b5-6f57-4c80-a84e-e067c3e8c97c None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:02.451344 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea502a6-c977-44dc-a49d-b802397b13d7 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:02.485956 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5460b5-6f57-4c80-a84e-e067c3e8c97c None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:02.489376 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1d78e8-c716-4d84-992f-fa4d55809da2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:02.490117 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1d78e8-c716-4d84-992f-fa4d55809da2 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:02.490506 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1d78e8-c716-4d84-992f-fa4d55809da2 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:02.490888 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1d78e8-c716-4d84-992f-fa4d55809da2 None admin] PATH_INFO: `/v3/groups/f5b13eb0527d4f748d667e2b1d087e54` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:02.492268 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1d78e8-c716-4d84-992f-fa4d55809da2 None admin] RBAC: Authorizing `identity:get_group(group_id=f5b13eb0527d4f748d667e2b1d087e54)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:02.493839 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1d78e8-c716-4d84-992f-fa4d55809da2 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:02.495616 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 176/373] 10.222.0.81 () {56 vars in 1180 bytes} [Wed Jul 20 22:26:02 2022] GET /identity/v3/groups/f5b13eb0527d4f748d667e2b1d087e54 => generated 226 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:02.496474 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1d78e8-c716-4d84-992f-fa4d55809da2 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:02.516551 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:02.524949 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:02.530902 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d709e09-e739-4744-9b60-3269d9eae6db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:02.532031 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d709e09-e739-4744-9b60-3269d9eae6db None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:02.537329 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d709e09-e739-4744-9b60-3269d9eae6db None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:02.537329 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d709e09-e739-4744-9b60-3269d9eae6db None admin] PATH_INFO: `/v3/projects/7165d638ee164d27903602ef884fcf03` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:02.537329 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d709e09-e739-4744-9b60-3269d9eae6db None admin] RBAC: Authorizing `identity:get_project(project_id=7165d638ee164d27903602ef884fcf03)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:02.537329 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d709e09-e739-4744-9b60-3269d9eae6db None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:02.537329 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 198/374] 10.222.0.81 () {56 vars in 1186 bytes} [Wed Jul 20 22:26:02 2022] GET /identity/v3/projects/7165d638ee164d27903602ef884fcf03 => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:02.538621 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d709e09-e739-4744-9b60-3269d9eae6db None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:02.605353 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1d78e8-c716-4d84-992f-fa4d55809da2 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:02.642810 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1e2ce7-6f5f-4883-9a61-bdbf8e077aff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:02.643945 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1e2ce7-6f5f-4883-9a61-bdbf8e077aff None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:02.644345 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1e2ce7-6f5f-4883-9a61-bdbf8e077aff None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:02.644698 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1e2ce7-6f5f-4883-9a61-bdbf8e077aff None admin] PATH_INFO: `/v3/projects/015b89c7582349b2a89f8bbaea747017` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:02.646181 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1e2ce7-6f5f-4883-9a61-bdbf8e077aff None admin] RBAC: Authorizing `identity:get_project(project_id=015b89c7582349b2a89f8bbaea747017)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:02.647516 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1e2ce7-6f5f-4883-9a61-bdbf8e077aff None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:02.649894 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 177/375] 10.222.0.81 () {56 vars in 1186 bytes} [Wed Jul 20 22:26:02 2022] GET /identity/v3/projects/015b89c7582349b2a89f8bbaea747017 => generated 344 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:02.650824 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1e2ce7-6f5f-4883-9a61-bdbf8e077aff None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:02.653345 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d709e09-e739-4744-9b60-3269d9eae6db None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:02.691418 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2a8a27-04d2-42fe-b56d-9f43648a9f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:02.692376 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2a8a27-04d2-42fe-b56d-9f43648a9f79 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:02.692838 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2a8a27-04d2-42fe-b56d-9f43648a9f79 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:02.693430 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2a8a27-04d2-42fe-b56d-9f43648a9f79 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:02.694481 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2a8a27-04d2-42fe-b56d-9f43648a9f79 None admin] RBAC: Adding query filter params (group.id=f5b13eb0527d4f748d667e2b1d087e54, role.id=559942c4de1744169f8b5e0404ecde6b, scope.project.id=7165d638ee164d27903602ef884fcf03) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:02.695557 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2a8a27-04d2-42fe-b56d-9f43648a9f79 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:02.697364 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2a8a27-04d2-42fe-b56d-9f43648a9f79 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:02.704070 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 199/376] 10.222.0.81 () {56 vars in 1376 bytes} [Wed Jul 20 22:26:02 2022] GET /identity/v3/role_assignments?group.id=f5b13eb0527d4f748d667e2b1d087e54&scope.project.id=7165d638ee164d27903602ef884fcf03&role.id=559942c4de1744169f8b5e0404ecde6b => generated 260 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:02.705251 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2a8a27-04d2-42fe-b56d-9f43648a9f79 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:02.745254 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:02.761497 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1e2ce7-6f5f-4883-9a61-bdbf8e077aff None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:02.784324 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:02.800240 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a998145-8c73-4618-91da-73ae1ae34c37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:02.801283 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a998145-8c73-4618-91da-73ae1ae34c37 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:02.801641 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a998145-8c73-4618-91da-73ae1ae34c37 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:02.802016 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a998145-8c73-4618-91da-73ae1ae34c37 None admin] PATH_INFO: `/v3/groups/f5b13eb0527d4f748d667e2b1d087e54` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:02.803428 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a998145-8c73-4618-91da-73ae1ae34c37 None admin] RBAC: Authorizing `identity:get_group(group_id=f5b13eb0527d4f748d667e2b1d087e54)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:02.805186 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a998145-8c73-4618-91da-73ae1ae34c37 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:02.807247 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 178/377] 10.222.0.81 () {56 vars in 1180 bytes} [Wed Jul 20 22:26:02 2022] GET /identity/v3/groups/f5b13eb0527d4f748d667e2b1d087e54 => generated 226 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:02.813018 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a998145-8c73-4618-91da-73ae1ae34c37 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:02.836287 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2a8a27-04d2-42fe-b56d-9f43648a9f79 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:02.880486 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12fed62-5d0f-485a-9b79-f90c7a7423aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:02.881790 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12fed62-5d0f-485a-9b79-f90c7a7423aa None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:02.882325 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12fed62-5d0f-485a-9b79-f90c7a7423aa None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:02.883370 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12fed62-5d0f-485a-9b79-f90c7a7423aa None admin] PATH_INFO: `/v3/groups/4610f6e120da448ab12a53899c5ee533` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:02.890988 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12fed62-5d0f-485a-9b79-f90c7a7423aa None admin] RBAC: Authorizing `identity:get_group(group_id=4610f6e120da448ab12a53899c5ee533)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:02.892678 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12fed62-5d0f-485a-9b79-f90c7a7423aa None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:02.894994 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 200/378] 10.222.0.81 () {56 vars in 1180 bytes} [Wed Jul 20 22:26:02 2022] GET /identity/v3/groups/4610f6e120da448ab12a53899c5ee533 => generated 229 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:02.900314 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12fed62-5d0f-485a-9b79-f90c7a7423aa None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:02.932131 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a998145-8c73-4618-91da-73ae1ae34c37 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:02.969741 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922d3db8-9096-4534-a9c4-8cf65c6c7824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:02.970764 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922d3db8-9096-4534-a9c4-8cf65c6c7824 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:02.970875 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922d3db8-9096-4534-a9c4-8cf65c6c7824 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:02.971223 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922d3db8-9096-4534-a9c4-8cf65c6c7824 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:02.972191 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922d3db8-9096-4534-a9c4-8cf65c6c7824 None admin] RBAC: Adding query filter params (group.id=f5b13eb0527d4f748d667e2b1d087e54, role.id=6d5a3e6a22fa4f4fa47afcfce0246a1f, scope.project.id=7165d638ee164d27903602ef884fcf03) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:02.972583 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922d3db8-9096-4534-a9c4-8cf65c6c7824 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:02.974145 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922d3db8-9096-4534-a9c4-8cf65c6c7824 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:02.980959 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 179/379] 10.222.0.81 () {56 vars in 1376 bytes} [Wed Jul 20 22:26:02 2022] GET /identity/v3/role_assignments?group.id=f5b13eb0527d4f748d667e2b1d087e54&scope.project.id=7165d638ee164d27903602ef884fcf03&role.id=6d5a3e6a22fa4f4fa47afcfce0246a1f => generated 260 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 22:26:03.021389 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199b58f5-89f5-4295-bc93-b92e378454c9 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:03.021732 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199b58f5-89f5-4295-bc93-b92e378454c9 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:03.021961 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199b58f5-89f5-4295-bc93-b92e378454c9 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:03.023759 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 180/380] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:26:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 22:26:03.029551 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c84a73f-1f61-4c7d-b00f-948bf018c397 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:03.029840 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c84a73f-1f61-4c7d-b00f-948bf018c397 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:03.030162 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c84a73f-1f61-4c7d-b00f-948bf018c397 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:03.032703 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12fed62-5d0f-485a-9b79-f90c7a7423aa None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:03.049726 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-8c84a73f-1f61-4c7d-b00f-948bf018c397 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:26:03.051344 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-8c84a73f-1f61-4c7d-b00f-948bf018c397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:26:03.066134 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:03.066768 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:03.077580 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc38249f-9f1f-46a0-bc1a-680bbcc1f303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:03.078772 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc38249f-9f1f-46a0-bc1a-680bbcc1f303 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:03.079291 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc38249f-9f1f-46a0-bc1a-680bbcc1f303 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:03.079737 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc38249f-9f1f-46a0-bc1a-680bbcc1f303 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:03.093132 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc38249f-9f1f-46a0-bc1a-680bbcc1f303 None admin] RBAC: Adding query filter params (group.id=4610f6e120da448ab12a53899c5ee533, role.id=d89b6c92d35a401a8517f1d47f6be843, scope.project.id=015b89c7582349b2a89f8bbaea747017) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:03.093132 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc38249f-9f1f-46a0-bc1a-680bbcc1f303 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:03.093132 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc38249f-9f1f-46a0-bc1a-680bbcc1f303 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:03.093132 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 201/381] 10.222.0.81 () {56 vars in 1376 bytes} [Wed Jul 20 22:26:02 2022] GET /identity/v3/role_assignments?group.id=4610f6e120da448ab12a53899c5ee533&scope.project.id=015b89c7582349b2a89f8bbaea747017&role.id=d89b6c92d35a401a8517f1d47f6be843 => generated 260 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:03.131456 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 181/382] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:26:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 22:26:03.155350 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc38249f-9f1f-46a0-bc1a-680bbcc1f303 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:03.290742 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc38249f-9f1f-46a0-bc1a-680bbcc1f303 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:03.331334 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3535d01c-f3dc-4807-b213-d2a1ed4d9f43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:03.332214 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3535d01c-f3dc-4807-b213-d2a1ed4d9f43 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:03.332505 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3535d01c-f3dc-4807-b213-d2a1ed4d9f43 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:03.332787 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3535d01c-f3dc-4807-b213-d2a1ed4d9f43 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:03.344623 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3535d01c-f3dc-4807-b213-d2a1ed4d9f43 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 22:26:03.353162 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-3535d01c-f3dc-4807-b213-d2a1ed4d9f43 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 22:26:03.353162 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:03.353162 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:03.353162 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:03.353162 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:03.353162 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:03.353162 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:03.353162 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:03.353162 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:03.353162 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:03.353162 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:03.353162 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:03.353162 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 22:26:03.353162 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 22:26:03.353162 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 22:26:03.353162 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 22:26:03.354151 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 22:26:03.354151 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 22:26:03.354151 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 22:26:03.354151 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 22:26:03.354151 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:03.354151 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:03.354151 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:03.354151 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:03.354151 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:03.354151 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:03.354151 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:03.354151 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:03.354151 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:03.354151 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:03.354151 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:03.354151 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:03.354929 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:03.354929 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:03.354929 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:03.354929 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:03.354929 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 22:26:03.354929 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 22:26:03.354929 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 22:26:03.354929 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 22:26:03.354929 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 22:26:03.354929 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 22:26:03.354929 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 22:26:03.354929 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:26:03.354929 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 202/383] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Jul 20 22:26:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:03.360415 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c84a73f-1f61-4c7d-b00f-948bf018c397 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:03.473636 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c84a73f-1f61-4c7d-b00f-948bf018c397 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:03.513556 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa393558-b535-40aa-9aab-1b186bf62d10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:03.514379 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa393558-b535-40aa-9aab-1b186bf62d10 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:03.514771 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa393558-b535-40aa-9aab-1b186bf62d10 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:03.515201 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa393558-b535-40aa-9aab-1b186bf62d10 None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:03.516309 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa393558-b535-40aa-9aab-1b186bf62d10 None admin] RBAC: Adding query filter params (name=member) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:03.516744 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa393558-b535-40aa-9aab-1b186bf62d10 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:03.518499 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa393558-b535-40aa-9aab-1b186bf62d10 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:03.530062 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 182/384] 10.222.0.81 () {56 vars in 1101 bytes} [Wed Jul 20 22:26:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:03.535780 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3535d01c-f3dc-4807-b213-d2a1ed4d9f43 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:03.603055 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:03.657187 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3535d01c-f3dc-4807-b213-d2a1ed4d9f43 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:03.706957 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b930d1ba-4404-41ce-8931-951bfe35341d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:03.707933 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b930d1ba-4404-41ce-8931-951bfe35341d None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:03.708221 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b930d1ba-4404-41ce-8931-951bfe35341d None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:03.708565 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b930d1ba-4404-41ce-8931-951bfe35341d None admin] PATH_INFO: `/v3/groups/f5b13eb0527d4f748d667e2b1d087e54` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:03.711786 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b930d1ba-4404-41ce-8931-951bfe35341d None admin] RBAC: Authorizing `identity:get_group(group_id=f5b13eb0527d4f748d667e2b1d087e54)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:03.713278 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b930d1ba-4404-41ce-8931-951bfe35341d None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:03.715301 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 203/385] 10.222.0.81 () {56 vars in 1180 bytes} [Wed Jul 20 22:26:03 2022] GET /identity/v3/groups/f5b13eb0527d4f748d667e2b1d087e54 => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:03.720731 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa393558-b535-40aa-9aab-1b186bf62d10 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:03.740284 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:03.839441 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa393558-b535-40aa-9aab-1b186bf62d10 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:03.879330 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38819357-eef4-40f9-adfb-34bb3a2ba6a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:03.880561 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38819357-eef4-40f9-adfb-34bb3a2ba6a9 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:03.881099 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38819357-eef4-40f9-adfb-34bb3a2ba6a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:03.881559 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38819357-eef4-40f9-adfb-34bb3a2ba6a9 None admin] PATH_INFO: `/v3/projects/da54f6860e1f43c29cf90271280bdcae` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:03.883569 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38819357-eef4-40f9-adfb-34bb3a2ba6a9 None admin] RBAC: Authorizing `identity:get_project(project_id=da54f6860e1f43c29cf90271280bdcae)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:03.885216 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38819357-eef4-40f9-adfb-34bb3a2ba6a9 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:03.887498 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 183/386] 10.222.0.81 () {56 vars in 1186 bytes} [Wed Jul 20 22:26:03 2022] GET /identity/v3/projects/da54f6860e1f43c29cf90271280bdcae => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:03.896053 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b930d1ba-4404-41ce-8931-951bfe35341d None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:03.916726 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f467075-9e30-4445-a3de-d85adb098307 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:03.917723 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f467075-9e30-4445-a3de-d85adb098307 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:03.918212 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f467075-9e30-4445-a3de-d85adb098307 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:03.919797 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 184/387] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:26:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 22:26:03.932530 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff68caf4-2537-44c7-ad41-253961d6e518 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:03.933131 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff68caf4-2537-44c7-ad41-253961d6e518 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:03.933738 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff68caf4-2537-44c7-ad41-253961d6e518 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:03.952468 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-ff68caf4-2537-44c7-ad41-253961d6e518 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:26:03.953986 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-ff68caf4-2537-44c7-ad41-253961d6e518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:26:04.074837 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 185/388] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:26:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 22:26:04.080167 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b930d1ba-4404-41ce-8931-951bfe35341d None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:04.104180 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:04.109300 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff68caf4-2537-44c7-ad41-253961d6e518 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:04.124317 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:04.133968 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39e01bf-05a7-4409-ab5c-de52014c446a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:04.134629 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39e01bf-05a7-4409-ab5c-de52014c446a None admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:04.134918 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39e01bf-05a7-4409-ab5c-de52014c446a None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:04.135223 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39e01bf-05a7-4409-ab5c-de52014c446a None admin] PATH_INFO: `/v3/projects/da54f6860e1f43c29cf90271280bdcae/groups/f5b13eb0527d4f748d667e2b1d087e54/roles/559942c4de1744169f8b5e0404ecde6b` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:04.139837 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39e01bf-05a7-4409-ab5c-de52014c446a None admin] RBAC: Authorizing `identity:create_grant(project_id=da54f6860e1f43c29cf90271280bdcae, group_id=f5b13eb0527d4f748d667e2b1d087e54, role_id=559942c4de1744169f8b5e0404ecde6b)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:04.142471 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39e01bf-05a7-4409-ab5c-de52014c446a None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:04.236519 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 204/389] 10.222.0.81 () {58 vars in 1442 bytes} [Wed Jul 20 22:26:03 2022] PUT /identity/v3/projects/da54f6860e1f43c29cf90271280bdcae/groups/f5b13eb0527d4f748d667e2b1d087e54/roles/559942c4de1744169f8b5e0404ecde6b => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:26:04.238542 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff68caf4-2537-44c7-ad41-253961d6e518 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:04.284224 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55359709-62f4-472c-9c45-bafa0cc27b5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:04.285090 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55359709-62f4-472c-9c45-bafa0cc27b5a None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:04.285380 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55359709-62f4-472c-9c45-bafa0cc27b5a None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:04.285661 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55359709-62f4-472c-9c45-bafa0cc27b5a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:04.305439 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55359709-62f4-472c-9c45-bafa0cc27b5a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 22:26:04.314726 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-55359709-62f4-472c-9c45-bafa0cc27b5a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 22:26:04.314726 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:04.314726 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:04.314726 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:04.314726 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:04.314726 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:04.314726 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:04.314726 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:04.314726 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:04.314726 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:04.314726 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:04.314726 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:04.314726 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 22:26:04.314726 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 22:26:04.314726 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 22:26:04.314726 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 22:26:04.315727 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 22:26:04.315727 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 22:26:04.315727 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 22:26:04.315727 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 22:26:04.315727 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:04.315727 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:04.315727 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:04.315727 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:04.315727 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:04.315727 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:04.315727 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:04.315727 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:04.315727 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:04.315727 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:04.315727 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:04.315727 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:04.316774 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:04.316774 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:04.316774 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:04.316774 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:04.316774 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 22:26:04.316774 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 22:26:04.316774 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 22:26:04.316774 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 22:26:04.316774 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 22:26:04.316774 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 22:26:04.316774 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 22:26:04.316774 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:26:04.316774 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 186/390] 10.222.0.81 () {56 vars in 1114 bytes} [Wed Jul 20 22:26:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 22:26:04.323126 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39e01bf-05a7-4409-ab5c-de52014c446a None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:04.449527 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39e01bf-05a7-4409-ab5c-de52014c446a None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:04.487604 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a401e9d6-c07f-4ff1-9a25-b495aadcb308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:04.488347 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a401e9d6-c07f-4ff1-9a25-b495aadcb308 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:04.488631 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a401e9d6-c07f-4ff1-9a25-b495aadcb308 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:04.488958 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a401e9d6-c07f-4ff1-9a25-b495aadcb308 None admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:04.489717 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a401e9d6-c07f-4ff1-9a25-b495aadcb308 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:04.490036 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a401e9d6-c07f-4ff1-9a25-b495aadcb308 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:04.491442 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a401e9d6-c07f-4ff1-9a25-b495aadcb308 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:04.502360 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 205/391] 10.222.0.81 () {56 vars in 1111 bytes} [Wed Jul 20 22:26:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:04.507625 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55359709-62f4-472c-9c45-bafa0cc27b5a None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:04.579523 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:04.622008 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55359709-62f4-472c-9c45-bafa0cc27b5a None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:04.661729 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec9e688-e081-4afe-883e-36ebae4670a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:04.662625 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec9e688-e081-4afe-883e-36ebae4670a2 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:04.662913 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec9e688-e081-4afe-883e-36ebae4670a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:04.663225 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec9e688-e081-4afe-883e-36ebae4670a2 None admin] PATH_INFO: `/v3/groups/f5b13eb0527d4f748d667e2b1d087e54` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:04.664459 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec9e688-e081-4afe-883e-36ebae4670a2 None admin] RBAC: Authorizing `identity:get_group(group_id=f5b13eb0527d4f748d667e2b1d087e54)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:04.665909 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec9e688-e081-4afe-883e-36ebae4670a2 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:04.668125 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 187/392] 10.222.0.81 () {56 vars in 1180 bytes} [Wed Jul 20 22:26:04 2022] GET /identity/v3/groups/f5b13eb0527d4f748d667e2b1d087e54 => generated 226 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:04.673258 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a401e9d6-c07f-4ff1-9a25-b495aadcb308 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:04.725737 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:04.793588 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a401e9d6-c07f-4ff1-9a25-b495aadcb308 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:04.834945 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf03ccfa-7cb4-4a8d-96c7-8c185b5a4ce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:04.835989 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf03ccfa-7cb4-4a8d-96c7-8c185b5a4ce0 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:04.836300 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf03ccfa-7cb4-4a8d-96c7-8c185b5a4ce0 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:04.836584 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf03ccfa-7cb4-4a8d-96c7-8c185b5a4ce0 None admin] PATH_INFO: `/v3/projects/da54f6860e1f43c29cf90271280bdcae` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:04.837955 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf03ccfa-7cb4-4a8d-96c7-8c185b5a4ce0 None admin] RBAC: Authorizing `identity:get_project(project_id=da54f6860e1f43c29cf90271280bdcae)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:04.839601 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf03ccfa-7cb4-4a8d-96c7-8c185b5a4ce0 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:04.841701 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 206/393] 10.222.0.81 () {56 vars in 1186 bytes} [Wed Jul 20 22:26:04 2022] GET /identity/v3/projects/da54f6860e1f43c29cf90271280bdcae => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:04.847146 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec9e688-e081-4afe-883e-36ebae4670a2 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:04.993607 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec9e688-e081-4afe-883e-36ebae4670a2 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:05.031857 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94964cb4-8921-468f-8bf2-aa8e4033d12e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:05.032855 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94964cb4-8921-468f-8bf2-aa8e4033d12e None admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:05.033141 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94964cb4-8921-468f-8bf2-aa8e4033d12e None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:05.033438 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94964cb4-8921-468f-8bf2-aa8e4033d12e None admin] PATH_INFO: `/v3/projects/da54f6860e1f43c29cf90271280bdcae/groups/f5b13eb0527d4f748d667e2b1d087e54/roles/6d5a3e6a22fa4f4fa47afcfce0246a1f` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:05.040983 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94964cb4-8921-468f-8bf2-aa8e4033d12e None admin] RBAC: Authorizing `identity:create_grant(project_id=da54f6860e1f43c29cf90271280bdcae, group_id=f5b13eb0527d4f748d667e2b1d087e54, role_id=6d5a3e6a22fa4f4fa47afcfce0246a1f)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:05.042804 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94964cb4-8921-468f-8bf2-aa8e4033d12e None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:05.050154 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:05.127736 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 188/394] 10.222.0.81 () {58 vars in 1442 bytes} [Wed Jul 20 22:26:04 2022] PUT /identity/v3/projects/da54f6860e1f43c29cf90271280bdcae/groups/f5b13eb0527d4f748d667e2b1d087e54/roles/6d5a3e6a22fa4f4fa47afcfce0246a1f => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 22:26:05.505362 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a539c1d6-d5e6-4f73-8ee4-7fc17deacd81 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:05.505747 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a539c1d6-d5e6-4f73-8ee4-7fc17deacd81 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:05.505924 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a539c1d6-d5e6-4f73-8ee4-7fc17deacd81 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:05.507837 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 207/395] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:26:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:26:05.513427 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb49b260-0cc9-4fef-a94b-469743ba8185 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:05.513751 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb49b260-0cc9-4fef-a94b-469743ba8185 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:05.514058 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb49b260-0cc9-4fef-a94b-469743ba8185 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:05.533630 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-eb49b260-0cc9-4fef-a94b-469743ba8185 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:26:05.534819 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-eb49b260-0cc9-4fef-a94b-469743ba8185 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:26:05.620715 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 189/396] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:26:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 22:26:05.660432 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a539c1d6-d5e6-4f73-8ee4-7fc17deacd81 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:05.792737 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a539c1d6-d5e6-4f73-8ee4-7fc17deacd81 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:05.804058 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:05.829033 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5834c6a7-c997-42d0-b6e8-32cde9b7e252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:05.829843 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5834c6a7-c997-42d0-b6e8-32cde9b7e252 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:05.830174 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5834c6a7-c997-42d0-b6e8-32cde9b7e252 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:05.830555 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5834c6a7-c997-42d0-b6e8-32cde9b7e252 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:05.845237 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5834c6a7-c997-42d0-b6e8-32cde9b7e252 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 22:26:05.854229 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-5834c6a7-c997-42d0-b6e8-32cde9b7e252 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 22:26:05.854229 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:05.854229 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:05.854229 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:05.854229 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:05.854229 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:05.854229 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:05.854229 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:05.854229 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:05.854229 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:05.854229 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:05.854229 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:05.854229 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 22:26:05.854229 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 22:26:05.854229 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 22:26:05.854229 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 22:26:05.855230 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 22:26:05.855230 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 22:26:05.855230 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 22:26:05.855230 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 22:26:05.855230 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:05.855230 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:05.855230 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:05.855230 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:05.855230 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:05.855230 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:05.855230 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:05.855230 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:05.855230 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:05.855230 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:05.855230 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:05.855230 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:05.856026 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:05.856026 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:05.856026 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:05.856026 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:05.856026 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 22:26:05.856026 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 22:26:05.856026 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 22:26:05.856026 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 22:26:05.856026 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 22:26:05.856026 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 22:26:05.856026 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 22:26:05.856026 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:26:05.856026 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 208/397] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Jul 20 22:26:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:05.861930 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb49b260-0cc9-4fef-a94b-469743ba8185 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:05.884638 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba84beb-d245-4ae2-b94a-8660e8cebbc7 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:05.884942 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba84beb-d245-4ae2-b94a-8660e8cebbc7 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:05.885194 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba84beb-d245-4ae2-b94a-8660e8cebbc7 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:05.886919 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 209/398] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:26:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 22:26:05.892163 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec55608-565d-497f-a4e1-4e071477d684 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:05.892457 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec55608-565d-497f-a4e1-4e071477d684 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:05.892744 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec55608-565d-497f-a4e1-4e071477d684 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:05.910796 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-3ec55608-565d-497f-a4e1-4e071477d684 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:26:05.912033 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-3ec55608-565d-497f-a4e1-4e071477d684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:26:05.991906 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 210/399] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:26:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 22:26:05.999381 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658fff32-ed74-4ba7-8dca-a694c26830d4 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:06.000066 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658fff32-ed74-4ba7-8dca-a694c26830d4 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:06.000556 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658fff32-ed74-4ba7-8dca-a694c26830d4 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:06.002931 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 211/400] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:26:05 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:26:06.008379 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86c0229-cd88-410b-9684-b4abc0dc41ce None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:06.008939 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86c0229-cd88-410b-9684-b4abc0dc41ce None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:06.009471 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86c0229-cd88-410b-9684-b4abc0dc41ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:06.020886 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb49b260-0cc9-4fef-a94b-469743ba8185 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:06.031420 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-d86c0229-cd88-410b-9684-b4abc0dc41ce None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:26:06.033307 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-d86c0229-cd88-410b-9684-b4abc0dc41ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:26:06.054467 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:06.064659 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b048fa76-d01d-4697-8f6f-bf7bb76c2564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:06.065632 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b048fa76-d01d-4697-8f6f-bf7bb76c2564 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:06.066374 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b048fa76-d01d-4697-8f6f-bf7bb76c2564 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:06.066665 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b048fa76-d01d-4697-8f6f-bf7bb76c2564 None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:06.067741 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b048fa76-d01d-4697-8f6f-bf7bb76c2564 None admin] RBAC: Adding query filter params (name=member) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:06.068249 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b048fa76-d01d-4697-8f6f-bf7bb76c2564 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:06.070151 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b048fa76-d01d-4697-8f6f-bf7bb76c2564 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:06.080009 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 190/401] 10.222.0.81 () {56 vars in 1101 bytes} [Wed Jul 20 22:26:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:06.081397 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b048fa76-d01d-4697-8f6f-bf7bb76c2564 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:06.115036 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 212/402] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:26:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 22:26:06.115036 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86c0229-cd88-410b-9684-b4abc0dc41ce None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:06.172835 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:06.215649 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b048fa76-d01d-4697-8f6f-bf7bb76c2564 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:06.229418 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86c0229-cd88-410b-9684-b4abc0dc41ce None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:06.250563 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becde03d-cd0d-4945-84d5-03b9335b16ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:06.251754 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becde03d-cd0d-4945-84d5-03b9335b16ba None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:06.252249 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becde03d-cd0d-4945-84d5-03b9335b16ba None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:06.252751 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becde03d-cd0d-4945-84d5-03b9335b16ba None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:06.264245 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becde03d-cd0d-4945-84d5-03b9335b16ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 22:26:06.266591 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcc39d3-1b57-49d8-b7a6-77b5f612ca07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:06.267534 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcc39d3-1b57-49d8-b7a6-77b5f612ca07 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:06.267822 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcc39d3-1b57-49d8-b7a6-77b5f612ca07 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:06.268106 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcc39d3-1b57-49d8-b7a6-77b5f612ca07 None admin] PATH_INFO: `/v3/groups/f5b13eb0527d4f748d667e2b1d087e54` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:06.269299 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcc39d3-1b57-49d8-b7a6-77b5f612ca07 None admin] RBAC: Authorizing `identity:get_group(group_id=f5b13eb0527d4f748d667e2b1d087e54)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:06.270756 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcc39d3-1b57-49d8-b7a6-77b5f612ca07 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:06.272410 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 213/403] 10.222.0.81 () {56 vars in 1180 bytes} [Wed Jul 20 22:26:06 2022] GET /identity/v3/groups/f5b13eb0527d4f748d667e2b1d087e54 => generated 226 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:06.273813 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcc39d3-1b57-49d8-b7a6-77b5f612ca07 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:06.275405 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-becde03d-cd0d-4945-84d5-03b9335b16ba None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 22:26:06.275405 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:06.275405 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:06.275405 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:06.275405 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:06.275405 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:06.275405 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:06.275405 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:06.275405 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:06.275405 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:06.275405 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:06.275405 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:06.275405 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 22:26:06.275405 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 22:26:06.275405 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 22:26:06.275405 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 22:26:06.276275 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 22:26:06.276275 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 22:26:06.276275 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 22:26:06.276275 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 22:26:06.276275 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:06.276275 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:06.276275 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:06.276275 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:06.276275 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:06.276275 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:06.276275 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:06.276275 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:06.276275 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:06.276275 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:06.276275 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:06.276275 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:06.277079 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:06.277079 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:06.277079 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:06.277079 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:06.277079 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 22:26:06.277079 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 22:26:06.277079 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 22:26:06.277079 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 22:26:06.277079 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 22:26:06.277079 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 22:26:06.277079 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 22:26:06.277079 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:26:06.279700 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 191/404] 10.222.0.81 () {56 vars in 1096 bytes} [Wed Jul 20 22:26:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:06.280698 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becde03d-cd0d-4945-84d5-03b9335b16ba None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:06.334570 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:06.383434 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:06.388522 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becde03d-cd0d-4945-84d5-03b9335b16ba None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:06.391735 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcc39d3-1b57-49d8-b7a6-77b5f612ca07 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:06.429410 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c1d28c-f8b5-4783-b6e5-cc963148e3b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:06.430350 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c1d28c-f8b5-4783-b6e5-cc963148e3b3 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:06.430749 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c1d28c-f8b5-4783-b6e5-cc963148e3b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:06.431150 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c1d28c-f8b5-4783-b6e5-cc963148e3b3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:06.433328 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cf18d2-f2f8-428c-ac7e-d9a5077a387b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:06.434684 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cf18d2-f2f8-428c-ac7e-d9a5077a387b None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:06.435188 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cf18d2-f2f8-428c-ac7e-d9a5077a387b None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:06.435662 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cf18d2-f2f8-428c-ac7e-d9a5077a387b None admin] PATH_INFO: `/v3/projects/7165d638ee164d27903602ef884fcf03` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:06.437313 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cf18d2-f2f8-428c-ac7e-d9a5077a387b None admin] RBAC: Authorizing `identity:get_project(project_id=7165d638ee164d27903602ef884fcf03)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:06.439119 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cf18d2-f2f8-428c-ac7e-d9a5077a387b None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:06.441748 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 192/405] 10.222.0.81 () {56 vars in 1186 bytes} [Wed Jul 20 22:26:06 2022] GET /identity/v3/projects/7165d638ee164d27903602ef884fcf03 => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:06.442527 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c1d28c-f8b5-4783-b6e5-cc963148e3b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 22:26:06.443143 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cf18d2-f2f8-428c-ac7e-d9a5077a387b None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:06.452610 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-c8c1d28c-f8b5-4783-b6e5-cc963148e3b3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 22:26:06.452610 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:06.452610 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:06.452610 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:06.452610 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:06.452610 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:06.452610 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:06.452610 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:06.452610 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:06.452610 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:06.452610 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:06.452610 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:06.452610 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 22:26:06.452610 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 22:26:06.452610 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 22:26:06.452610 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 22:26:06.453522 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 22:26:06.453522 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 22:26:06.453522 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 22:26:06.453522 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 22:26:06.453522 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:06.453522 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:06.453522 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:06.453522 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:06.453522 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:06.453522 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:06.453522 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:06.453522 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:06.453522 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:06.453522 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:06.453522 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:06.453522 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:06.454314 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:06.454314 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:06.454314 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:06.454314 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:06.454314 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 22:26:06.454314 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 22:26:06.454314 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 22:26:06.454314 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 22:26:06.454314 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 22:26:06.454314 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 22:26:06.454314 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 22:26:06.454314 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:26:06.454314 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 214/406] 10.222.0.81 () {56 vars in 1114 bytes} [Wed Jul 20 22:26:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:06.454955 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c1d28c-f8b5-4783-b6e5-cc963148e3b3 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:06.577463 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cf18d2-f2f8-428c-ac7e-d9a5077a387b None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:06.581370 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c1d28c-f8b5-4783-b6e5-cc963148e3b3 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:06.596302 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:06.615515 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc665f6-f683-486d-8102-c2829c06cdc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:06.617058 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc665f6-f683-486d-8102-c2829c06cdc4 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:06.617548 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc665f6-f683-486d-8102-c2829c06cdc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:06.618046 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc665f6-f683-486d-8102-c2829c06cdc4 None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:06.619126 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc665f6-f683-486d-8102-c2829c06cdc4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:06.619634 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc665f6-f683-486d-8102-c2829c06cdc4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:06.621397 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc665f6-f683-486d-8102-c2829c06cdc4 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:06.624287 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4784aaa-5ef0-4388-8191-e412e4082330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:06.638455 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4784aaa-5ef0-4388-8191-e412e4082330 None admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:06.638455 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4784aaa-5ef0-4388-8191-e412e4082330 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:06.638455 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4784aaa-5ef0-4388-8191-e412e4082330 None admin] PATH_INFO: `/v3/projects/7165d638ee164d27903602ef884fcf03/groups/f5b13eb0527d4f748d667e2b1d087e54/roles/559942c4de1744169f8b5e0404ecde6b` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:06.638455 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4784aaa-5ef0-4388-8191-e412e4082330 None admin] RBAC: Authorizing `identity:create_grant(project_id=7165d638ee164d27903602ef884fcf03, group_id=f5b13eb0527d4f748d667e2b1d087e54, role_id=559942c4de1744169f8b5e0404ecde6b)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:06.638455 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4784aaa-5ef0-4388-8191-e412e4082330 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:06.638816 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 193/407] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Jul 20 22:26:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:06.638816 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc665f6-f683-486d-8102-c2829c06cdc4 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:06.742280 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc665f6-f683-486d-8102-c2829c06cdc4 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:06.768942 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 215/408] 10.222.0.81 () {58 vars in 1442 bytes} [Wed Jul 20 22:26:06 2022] PUT /identity/v3/projects/7165d638ee164d27903602ef884fcf03/groups/f5b13eb0527d4f748d667e2b1d087e54/roles/559942c4de1744169f8b5e0404ecde6b => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:26:06.768942 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4784aaa-5ef0-4388-8191-e412e4082330 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:06.800208 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6299aa93-983b-4df6-826f-252127a45c39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:06.801930 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6299aa93-983b-4df6-826f-252127a45c39 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:06.802236 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6299aa93-983b-4df6-826f-252127a45c39 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:06.802505 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6299aa93-983b-4df6-826f-252127a45c39 None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:06.803290 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6299aa93-983b-4df6-826f-252127a45c39 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:06.808616 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6299aa93-983b-4df6-826f-252127a45c39 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:06.808616 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6299aa93-983b-4df6-826f-252127a45c39 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:06.815653 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:06.827858 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 194/409] 10.222.0.81 () {56 vars in 1111 bytes} [Wed Jul 20 22:26:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:06.834834 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6299aa93-983b-4df6-826f-252127a45c39 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:06.940384 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4784aaa-5ef0-4388-8191-e412e4082330 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:06.941704 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:06.983999 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c1b50f-212b-4a8f-8c24-310eee0cd5f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:06.985232 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c1b50f-212b-4a8f-8c24-310eee0cd5f0 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:06.985719 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c1b50f-212b-4a8f-8c24-310eee0cd5f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:06.986239 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c1b50f-212b-4a8f-8c24-310eee0cd5f0 None admin] PATH_INFO: `/v3/groups/4610f6e120da448ab12a53899c5ee533` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:06.988290 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c1b50f-212b-4a8f-8c24-310eee0cd5f0 None admin] RBAC: Authorizing `identity:get_group(group_id=4610f6e120da448ab12a53899c5ee533)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:06.991922 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c1b50f-212b-4a8f-8c24-310eee0cd5f0 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:06.999111 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 216/410] 10.222.0.81 () {56 vars in 1180 bytes} [Wed Jul 20 22:26:06 2022] GET /identity/v3/groups/4610f6e120da448ab12a53899c5ee533 => generated 229 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:07.001255 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c1b50f-212b-4a8f-8c24-310eee0cd5f0 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:07.051114 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6299aa93-983b-4df6-826f-252127a45c39 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:07.096761 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a17119-b27d-46ac-bcfa-6d70cc02a39c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:07.098395 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a17119-b27d-46ac-bcfa-6d70cc02a39c None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:07.098923 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a17119-b27d-46ac-bcfa-6d70cc02a39c None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:07.099422 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a17119-b27d-46ac-bcfa-6d70cc02a39c None admin] PATH_INFO: `/v3/groups/f5b13eb0527d4f748d667e2b1d087e54` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:07.102502 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a17119-b27d-46ac-bcfa-6d70cc02a39c None admin] RBAC: Authorizing `identity:get_group(group_id=f5b13eb0527d4f748d667e2b1d087e54)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:07.103465 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:07.104834 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a17119-b27d-46ac-bcfa-6d70cc02a39c None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:07.107540 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 195/411] 10.222.0.81 () {56 vars in 1180 bytes} [Wed Jul 20 22:26:06 2022] GET /identity/v3/groups/f5b13eb0527d4f748d667e2b1d087e54 => generated 226 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:07.110074 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aa54d0-82b5-4e54-ac83-d4190686d62c None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:07.110567 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aa54d0-82b5-4e54-ac83-d4190686d62c None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:07.117116 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7aa54d0-82b5-4e54-ac83-d4190686d62c None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:07.117116 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 196/412] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:26:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 22:26:07.117116 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aa54d0-82b5-4e54-ac83-d4190686d62c None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:07.119643 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c1b50f-212b-4a8f-8c24-310eee0cd5f0 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:07.154639 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e056298-36a2-44ba-bbc9-b492f20129bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:07.155562 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e056298-36a2-44ba-bbc9-b492f20129bb None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:07.155845 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e056298-36a2-44ba-bbc9-b492f20129bb None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:07.156179 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e056298-36a2-44ba-bbc9-b492f20129bb None admin] PATH_INFO: `/v3/projects/015b89c7582349b2a89f8bbaea747017` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:07.157296 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e056298-36a2-44ba-bbc9-b492f20129bb None admin] RBAC: Authorizing `identity:get_project(project_id=015b89c7582349b2a89f8bbaea747017)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:07.158771 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e056298-36a2-44ba-bbc9-b492f20129bb None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:07.165727 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 217/413] 10.222.0.81 () {56 vars in 1186 bytes} [Wed Jul 20 22:26:06 2022] GET /identity/v3/projects/015b89c7582349b2a89f8bbaea747017 => generated 344 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 22:26:07.165727 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecd9433-9b5e-4a9e-a94f-7ecbd9d052df None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:07.165727 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecd9433-9b5e-4a9e-a94f-7ecbd9d052df None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:07.165727 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecd9433-9b5e-4a9e-a94f-7ecbd9d052df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:07.196941 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-0ecd9433-9b5e-4a9e-a94f-7ecbd9d052df None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:26:07.199172 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-0ecd9433-9b5e-4a9e-a94f-7ecbd9d052df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:26:07.243639 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7aa54d0-82b5-4e54-ac83-d4190686d62c None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:07.253917 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:07.281633 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ac13ba-c538-48f1-9aff-2b0f5f72d380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:07.283137 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ac13ba-c538-48f1-9aff-2b0f5f72d380 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:07.283617 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ac13ba-c538-48f1-9aff-2b0f5f72d380 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:07.284594 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ac13ba-c538-48f1-9aff-2b0f5f72d380 None admin] PATH_INFO: `/v3/projects/7165d638ee164d27903602ef884fcf03` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:07.285689 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ac13ba-c538-48f1-9aff-2b0f5f72d380 None admin] RBAC: Authorizing `identity:get_project(project_id=7165d638ee164d27903602ef884fcf03)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:07.287556 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ac13ba-c538-48f1-9aff-2b0f5f72d380 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:07.290183 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 218/414] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:26:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 22:26:07.291246 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecd9433-9b5e-4a9e-a94f-7ecbd9d052df None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:07.295558 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 197/415] 10.222.0.81 () {56 vars in 1186 bytes} [Wed Jul 20 22:26:07 2022] GET /identity/v3/projects/7165d638ee164d27903602ef884fcf03 => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 22:26:07.296191 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ac13ba-c538-48f1-9aff-2b0f5f72d380 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:07.414744 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:07.434145 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecd9433-9b5e-4a9e-a94f-7ecbd9d052df None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:07.437868 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ac13ba-c538-48f1-9aff-2b0f5f72d380 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:07.470063 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ba26a0-bc62-4261-bb8d-9147e5cc5071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:07.470880 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ba26a0-bc62-4261-bb8d-9147e5cc5071 None admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:07.471245 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ba26a0-bc62-4261-bb8d-9147e5cc5071 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:07.471451 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ba26a0-bc62-4261-bb8d-9147e5cc5071 None admin] PATH_INFO: `/v3/projects/015b89c7582349b2a89f8bbaea747017/groups/4610f6e120da448ab12a53899c5ee533/roles/d89b6c92d35a401a8517f1d47f6be843` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:07.473403 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ba26a0-bc62-4261-bb8d-9147e5cc5071 None admin] RBAC: Authorizing `identity:create_grant(project_id=015b89c7582349b2a89f8bbaea747017, group_id=4610f6e120da448ab12a53899c5ee533, role_id=d89b6c92d35a401a8517f1d47f6be843)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:07.474878 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ba26a0-bc62-4261-bb8d-9147e5cc5071 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:07.481666 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1659efe6-b09b-4ed7-a269-0789e1a7f588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:07.483322 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1659efe6-b09b-4ed7-a269-0789e1a7f588 None admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:07.483829 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1659efe6-b09b-4ed7-a269-0789e1a7f588 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:07.484329 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1659efe6-b09b-4ed7-a269-0789e1a7f588 None admin] PATH_INFO: `/v3/projects/7165d638ee164d27903602ef884fcf03/groups/f5b13eb0527d4f748d667e2b1d087e54/roles/6d5a3e6a22fa4f4fa47afcfce0246a1f` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:07.486757 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1659efe6-b09b-4ed7-a269-0789e1a7f588 None admin] RBAC: Authorizing `identity:create_grant(project_id=7165d638ee164d27903602ef884fcf03, group_id=f5b13eb0527d4f748d667e2b1d087e54, role_id=6d5a3e6a22fa4f4fa47afcfce0246a1f)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:07.488422 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1659efe6-b09b-4ed7-a269-0789e1a7f588 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:07.555951 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 219/416] 10.222.0.81 () {58 vars in 1442 bytes} [Wed Jul 20 22:26:07 2022] PUT /identity/v3/projects/015b89c7582349b2a89f8bbaea747017/groups/4610f6e120da448ab12a53899c5ee533/roles/d89b6c92d35a401a8517f1d47f6be843 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:26:07.557264 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ba26a0-bc62-4261-bb8d-9147e5cc5071 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:07.602064 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 198/417] 10.222.0.81 () {58 vars in 1442 bytes} [Wed Jul 20 22:26:07 2022] PUT /identity/v3/projects/7165d638ee164d27903602ef884fcf03/groups/f5b13eb0527d4f748d667e2b1d087e54/roles/6d5a3e6a22fa4f4fa47afcfce0246a1f => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:26:07.672553 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac912e7-c759-4aff-bacf-447a92400b65 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:07.672876 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac912e7-c759-4aff-bacf-447a92400b65 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:07.673127 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac912e7-c759-4aff-bacf-447a92400b65 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:07.674789 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 199/418] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:26:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:26:07.680481 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20aa12c5-5fa9-47d7-a441-33b06b139009 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:07.680768 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20aa12c5-5fa9-47d7-a441-33b06b139009 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:07.681046 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20aa12c5-5fa9-47d7-a441-33b06b139009 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:07.695181 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ba26a0-bc62-4261-bb8d-9147e5cc5071 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:07.699545 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-20aa12c5-5fa9-47d7-a441-33b06b139009 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:26:07.700759 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-20aa12c5-5fa9-47d7-a441-33b06b139009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:26:07.740857 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:07.747043 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9440a8e-18e9-45db-88be-25a12fa7f1e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:07.748276 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9440a8e-18e9-45db-88be-25a12fa7f1e5 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:07.748561 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9440a8e-18e9-45db-88be-25a12fa7f1e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:07.748838 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9440a8e-18e9-45db-88be-25a12fa7f1e5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:07.762375 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9440a8e-18e9-45db-88be-25a12fa7f1e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 22:26:07.770932 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-e9440a8e-18e9-45db-88be-25a12fa7f1e5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 22:26:07.770932 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:07.770932 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:07.770932 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:07.770932 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:07.770932 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:07.770932 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:07.770932 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:07.770932 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:07.770932 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:07.770932 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:07.770932 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:07.770932 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 22:26:07.770932 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 22:26:07.770932 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 22:26:07.770932 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 22:26:07.771927 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 22:26:07.771927 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 22:26:07.771927 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 22:26:07.771927 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 22:26:07.771927 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:07.771927 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:07.771927 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:07.771927 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:07.771927 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:07.771927 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:07.771927 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:07.771927 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:07.771927 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:07.771927 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:07.771927 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:07.771927 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:07.773320 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:07.773320 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:07.773320 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:07.773320 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:07.773320 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 22:26:07.773320 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 22:26:07.773320 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 22:26:07.773320 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 22:26:07.773320 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 22:26:07.773320 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 22:26:07.773320 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 22:26:07.773320 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:26:07.778487 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 220/419] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Jul 20 22:26:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:07.779488 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9440a8e-18e9-45db-88be-25a12fa7f1e5 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:07.784228 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:07.802449 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 200/420] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:26:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 22:26:07.862288 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20aa12c5-5fa9-47d7-a441-33b06b139009 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:07.939087 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9440a8e-18e9-45db-88be-25a12fa7f1e5 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:07.976490 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0b528d-6022-4b37-b9a6-5a6d1ebcb336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:07.977152 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20aa12c5-5fa9-47d7-a441-33b06b139009 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:07.977597 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0b528d-6022-4b37-b9a6-5a6d1ebcb336 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:07.977597 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0b528d-6022-4b37-b9a6-5a6d1ebcb336 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:07.977852 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0b528d-6022-4b37-b9a6-5a6d1ebcb336 None admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:07.978661 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0b528d-6022-4b37-b9a6-5a6d1ebcb336 None admin] RBAC: Adding query filter params (name=member) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:07.978982 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0b528d-6022-4b37-b9a6-5a6d1ebcb336 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:07.980517 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0b528d-6022-4b37-b9a6-5a6d1ebcb336 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:07.989988 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 221/421] 10.222.0.81 () {56 vars in 1101 bytes} [Wed Jul 20 22:26:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:07.996343 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0b528d-6022-4b37-b9a6-5a6d1ebcb336 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:08.013242 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da07af04-4c58-4089-9944-bddbf746bbc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:08.014094 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da07af04-4c58-4089-9944-bddbf746bbc0 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:08.014385 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da07af04-4c58-4089-9944-bddbf746bbc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:08.014667 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da07af04-4c58-4089-9944-bddbf746bbc0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:08.025409 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da07af04-4c58-4089-9944-bddbf746bbc0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 22:26:08.034126 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-da07af04-4c58-4089-9944-bddbf746bbc0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 22:26:08.034126 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:08.034126 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:08.034126 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:08.034126 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:08.034126 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:08.034126 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:08.034126 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:08.034126 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:08.034126 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:08.034126 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:08.034126 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:08.034126 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 22:26:08.034126 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 22:26:08.034126 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 22:26:08.034126 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 22:26:08.034999 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 22:26:08.034999 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 22:26:08.034999 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 22:26:08.034999 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 22:26:08.034999 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:08.034999 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:08.034999 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:08.034999 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:08.034999 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:08.034999 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:08.034999 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:08.034999 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:08.034999 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:08.034999 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:08.034999 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:08.034999 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:08.036286 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:08.036286 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:08.036286 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:08.036286 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:08.036286 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 22:26:08.036286 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 22:26:08.036286 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 22:26:08.036286 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 22:26:08.036286 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 22:26:08.036286 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 22:26:08.036286 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 22:26:08.036286 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:26:08.036286 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 201/422] 10.222.0.81 () {56 vars in 1114 bytes} [Wed Jul 20 22:26:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:08.041330 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da07af04-4c58-4089-9944-bddbf746bbc0 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:08.075427 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:08.094838 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:08.109362 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0b528d-6022-4b37-b9a6-5a6d1ebcb336 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:08.146314 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264bfe36-0133-4337-a728-35fedb4f9fa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:08.147079 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264bfe36-0133-4337-a728-35fedb4f9fa7 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:08.147403 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264bfe36-0133-4337-a728-35fedb4f9fa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:08.147690 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264bfe36-0133-4337-a728-35fedb4f9fa7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:08.244850 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264bfe36-0133-4337-a728-35fedb4f9fa7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:08.246336 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264bfe36-0133-4337-a728-35fedb4f9fa7 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:08.248857 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da07af04-4c58-4089-9944-bddbf746bbc0 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:08.327495 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:08.353749 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 222/423] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Jul 20 22:26:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 20 22:26:08.356302 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905957c7-f3c4-4d4c-a687-01bda248a39e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:08.356999 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905957c7-f3c4-4d4c-a687-01bda248a39e None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:08.357283 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905957c7-f3c4-4d4c-a687-01bda248a39e None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:08.357558 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905957c7-f3c4-4d4c-a687-01bda248a39e None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:08.358319 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905957c7-f3c4-4d4c-a687-01bda248a39e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:08.358618 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905957c7-f3c4-4d4c-a687-01bda248a39e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:08.359273 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264bfe36-0133-4337-a728-35fedb4f9fa7 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:08.360022 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905957c7-f3c4-4d4c-a687-01bda248a39e None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:08.382979 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 202/424] 10.222.0.81 () {56 vars in 1111 bytes} [Wed Jul 20 22:26:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:08.391989 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905957c7-f3c4-4d4c-a687-01bda248a39e None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:08.464113 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264bfe36-0133-4337-a728-35fedb4f9fa7 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:08.504891 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0c9d14-d1ee-49ff-8e07-c66edebb27cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:08.505698 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0c9d14-d1ee-49ff-8e07-c66edebb27cf None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:08.505987 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0c9d14-d1ee-49ff-8e07-c66edebb27cf None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:08.506522 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0c9d14-d1ee-49ff-8e07-c66edebb27cf None admin] PATH_INFO: `/v3/projects/da54f6860e1f43c29cf90271280bdcae` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:08.509007 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0c9d14-d1ee-49ff-8e07-c66edebb27cf None admin] RBAC: Authorizing `identity:get_project(project_id=da54f6860e1f43c29cf90271280bdcae)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:08.510663 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0c9d14-d1ee-49ff-8e07-c66edebb27cf None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:08.520132 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 223/425] 10.222.0.81 () {56 vars in 1186 bytes} [Wed Jul 20 22:26:08 2022] GET /identity/v3/projects/da54f6860e1f43c29cf90271280bdcae => generated 298 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 22:26:08.521693 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0c9d14-d1ee-49ff-8e07-c66edebb27cf None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:08.591460 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:08.594735 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:08.622259 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905957c7-f3c4-4d4c-a687-01bda248a39e None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:08.659262 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6873a64-7756-4cb1-9a15-ed2a403e1253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:08.659930 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6873a64-7756-4cb1-9a15-ed2a403e1253 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:08.660220 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6873a64-7756-4cb1-9a15-ed2a403e1253 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:08.660501 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6873a64-7756-4cb1-9a15-ed2a403e1253 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:08.697125 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0c9d14-d1ee-49ff-8e07-c66edebb27cf None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:08.730839 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6873a64-7756-4cb1-9a15-ed2a403e1253 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:08.732361 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6873a64-7756-4cb1-9a15-ed2a403e1253 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:08.811712 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540234fc-d277-4306-a216-69d3f4306f0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:08.812613 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540234fc-d277-4306-a216-69d3f4306f0c None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:08.812904 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540234fc-d277-4306-a216-69d3f4306f0c None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:08.813187 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540234fc-d277-4306-a216-69d3f4306f0c None admin] PATH_INFO: `/v3/groups/f5b13eb0527d4f748d667e2b1d087e54` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:08.814440 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540234fc-d277-4306-a216-69d3f4306f0c None admin] RBAC: Authorizing `identity:get_group(group_id=f5b13eb0527d4f748d667e2b1d087e54)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:08.815923 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540234fc-d277-4306-a216-69d3f4306f0c None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:08.818720 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 224/426] 10.222.0.81 () {56 vars in 1180 bytes} [Wed Jul 20 22:26:08 2022] GET /identity/v3/groups/f5b13eb0527d4f748d667e2b1d087e54 => generated 226 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 22:26:08.824468 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540234fc-d277-4306-a216-69d3f4306f0c None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:08.827751 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:08.838079 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 203/427] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Jul 20 22:26:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 22:26:08.843557 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6873a64-7756-4cb1-9a15-ed2a403e1253 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:08.951458 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6873a64-7756-4cb1-9a15-ed2a403e1253 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:08.989535 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bab60e-b1a6-45fd-ad2f-dce9333c9aa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:08.990439 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bab60e-b1a6-45fd-ad2f-dce9333c9aa5 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:08.990732 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bab60e-b1a6-45fd-ad2f-dce9333c9aa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:08.991057 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bab60e-b1a6-45fd-ad2f-dce9333c9aa5 None admin] PATH_INFO: `/v3/projects/da54f6860e1f43c29cf90271280bdcae` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:08.992311 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bab60e-b1a6-45fd-ad2f-dce9333c9aa5 None admin] RBAC: Authorizing `identity:get_project(project_id=da54f6860e1f43c29cf90271280bdcae)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:08.993764 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bab60e-b1a6-45fd-ad2f-dce9333c9aa5 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:08.995717 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 204/428] 10.222.0.81 () {56 vars in 1186 bytes} [Wed Jul 20 22:26:08 2022] GET /identity/v3/projects/da54f6860e1f43c29cf90271280bdcae => generated 298 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 22:26:09.001204 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bab60e-b1a6-45fd-ad2f-dce9333c9aa5 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:09.071492 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:09.114164 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bab60e-b1a6-45fd-ad2f-dce9333c9aa5 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:09.152817 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b0f49c-9b7a-49f4-b4eb-45b7be945be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:09.153720 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b0f49c-9b7a-49f4-b4eb-45b7be945be8 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:09.154016 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b0f49c-9b7a-49f4-b4eb-45b7be945be8 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:09.154306 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b0f49c-9b7a-49f4-b4eb-45b7be945be8 None admin] PATH_INFO: `/v3/groups/f5b13eb0527d4f748d667e2b1d087e54` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:09.155700 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b0f49c-9b7a-49f4-b4eb-45b7be945be8 None admin] RBAC: Authorizing `identity:get_group(group_id=f5b13eb0527d4f748d667e2b1d087e54)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:09.157351 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b0f49c-9b7a-49f4-b4eb-45b7be945be8 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:09.159349 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 205/429] 10.222.0.81 () {56 vars in 1180 bytes} [Wed Jul 20 22:26:09 2022] GET /identity/v3/groups/f5b13eb0527d4f748d667e2b1d087e54 => generated 226 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 22:26:09.165978 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b0f49c-9b7a-49f4-b4eb-45b7be945be8 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:09.235167 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540234fc-d277-4306-a216-69d3f4306f0c None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:09.243442 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:09.301778 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7805bf28-b9e4-446f-8bf6-fb4791a8931d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:09.302444 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7805bf28-b9e4-446f-8bf6-fb4791a8931d None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:09.302739 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7805bf28-b9e4-446f-8bf6-fb4791a8931d None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:09.303048 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7805bf28-b9e4-446f-8bf6-fb4791a8931d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:09.303878 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b0f49c-9b7a-49f4-b4eb-45b7be945be8 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:09.303992 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7805bf28-b9e4-446f-8bf6-fb4791a8931d None admin] RBAC: Adding query filter params (group.id=f5b13eb0527d4f748d667e2b1d087e54, role.id=559942c4de1744169f8b5e0404ecde6b, scope.project.id=da54f6860e1f43c29cf90271280bdcae) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:09.304233 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7805bf28-b9e4-446f-8bf6-fb4791a8931d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:09.305854 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7805bf28-b9e4-446f-8bf6-fb4791a8931d None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:09.313310 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 225/430] 10.222.0.81 () {56 vars in 1376 bytes} [Wed Jul 20 22:26:08 2022] GET /identity/v3/role_assignments?group.id=f5b13eb0527d4f748d667e2b1d087e54&scope.project.id=da54f6860e1f43c29cf90271280bdcae&role.id=559942c4de1744169f8b5e0404ecde6b => generated 611 bytes in 489 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 22:26:09.314564 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b99313-ba70-4d99-b9ab-eb068352dc0c None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:09.314863 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b99313-ba70-4d99-b9ab-eb068352dc0c None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:09.315180 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b99313-ba70-4d99-b9ab-eb068352dc0c None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:09.316692 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 226/431] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:26:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:26:09.332876 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725f79f2-28e5-4e53-bc4a-4bba81119935 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:09.333086 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725f79f2-28e5-4e53-bc4a-4bba81119935 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:09.333368 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725f79f2-28e5-4e53-bc4a-4bba81119935 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:09.352615 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-725f79f2-28e5-4e53-bc4a-4bba81119935 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:26:09.354133 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-725f79f2-28e5-4e53-bc4a-4bba81119935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:26:09.361672 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:09.405236 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b2663c-b711-498a-9a93-c0fc0398b07a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:09.405936 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b2663c-b711-498a-9a93-c0fc0398b07a None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:09.406232 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b2663c-b711-498a-9a93-c0fc0398b07a None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:09.406519 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b2663c-b711-498a-9a93-c0fc0398b07a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:09.407407 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b2663c-b711-498a-9a93-c0fc0398b07a None admin] RBAC: Adding query filter params (group.id=f5b13eb0527d4f748d667e2b1d087e54, role.id=6d5a3e6a22fa4f4fa47afcfce0246a1f, scope.project.id=da54f6860e1f43c29cf90271280bdcae) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:09.407728 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b2663c-b711-498a-9a93-c0fc0398b07a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:09.409195 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b2663c-b711-498a-9a93-c0fc0398b07a None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:09.433348 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 206/432] 10.222.0.81 () {56 vars in 1376 bytes} [Wed Jul 20 22:26:09 2022] GET /identity/v3/role_assignments?group.id=f5b13eb0527d4f748d667e2b1d087e54&scope.project.id=da54f6860e1f43c29cf90271280bdcae&role.id=6d5a3e6a22fa4f4fa47afcfce0246a1f => generated 611 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 22:26:09.434417 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 227/433] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:26:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 22:26:09.473359 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b2663c-b711-498a-9a93-c0fc0398b07a None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:09.594118 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b2663c-b711-498a-9a93-c0fc0398b07a None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:09.631249 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa0dc65-eaf9-450b-be07-1e0e9a0337e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:09.632117 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa0dc65-eaf9-450b-be07-1e0e9a0337e4 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:09.632409 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa0dc65-eaf9-450b-be07-1e0e9a0337e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:09.632692 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa0dc65-eaf9-450b-be07-1e0e9a0337e4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:09.643550 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa0dc65-eaf9-450b-be07-1e0e9a0337e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 22:26:09.652395 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-0aa0dc65-eaf9-450b-be07-1e0e9a0337e4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 22:26:09.652395 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:09.652395 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:09.652395 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:09.652395 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:09.652395 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:09.652395 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:09.652395 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:09.652395 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:09.652395 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:09.652395 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:09.652395 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:09.652395 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 22:26:09.652395 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 22:26:09.652395 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 22:26:09.652395 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 22:26:09.653309 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 22:26:09.653309 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 22:26:09.653309 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 22:26:09.653309 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 22:26:09.653309 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:09.653309 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:09.653309 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:09.653309 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:09.653309 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:09.653309 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:09.653309 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:09.653309 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:09.653309 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:09.653309 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:09.653309 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:09.653309 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:09.654132 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:09.654132 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:09.654132 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:09.654132 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:09.654132 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 22:26:09.654132 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 22:26:09.654132 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 22:26:09.654132 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 22:26:09.654132 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 22:26:09.654132 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 22:26:09.654132 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 22:26:09.654132 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:26:09.654132 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 207/434] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Jul 20 22:26:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:09.661157 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725f79f2-28e5-4e53-bc4a-4bba81119935 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:09.776457 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:09.785034 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725f79f2-28e5-4e53-bc4a-4bba81119935 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:09.821536 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f80bc1-07f6-497e-90fb-1c066e092f0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:09.822586 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f80bc1-07f6-497e-90fb-1c066e092f0c None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:09.823111 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f80bc1-07f6-497e-90fb-1c066e092f0c None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:09.823593 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f80bc1-07f6-497e-90fb-1c066e092f0c None admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:09.824544 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f80bc1-07f6-497e-90fb-1c066e092f0c None admin] RBAC: Adding query filter params (name=member) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:09.825201 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f80bc1-07f6-497e-90fb-1c066e092f0c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:09.826788 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f80bc1-07f6-497e-90fb-1c066e092f0c None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:09.838059 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 228/435] 10.222.0.81 () {56 vars in 1101 bytes} [Wed Jul 20 22:26:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:09.844037 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa0dc65-eaf9-450b-be07-1e0e9a0337e4 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:09.898215 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:09.953249 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa0dc65-eaf9-450b-be07-1e0e9a0337e4 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:09.989681 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402424c5-37cb-4338-8e29-a6e7e268f827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:09.991221 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402424c5-37cb-4338-8e29-a6e7e268f827 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:09.991712 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402424c5-37cb-4338-8e29-a6e7e268f827 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:09.992201 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402424c5-37cb-4338-8e29-a6e7e268f827 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:10.064230 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402424c5-37cb-4338-8e29-a6e7e268f827 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:10.067004 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabe1640-bbb6-4ba1-bf42-2839bbf457e4 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:10.067378 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabe1640-bbb6-4ba1-bf42-2839bbf457e4 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:10.067662 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabe1640-bbb6-4ba1-bf42-2839bbf457e4 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:10.069686 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 229/436] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:26:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 22:26:10.069961 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402424c5-37cb-4338-8e29-a6e7e268f827 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:10.075421 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460f8629-de58-4aed-9460-55d81a0f1787 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:10.075715 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460f8629-de58-4aed-9460-55d81a0f1787 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:10.075997 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460f8629-de58-4aed-9460-55d81a0f1787 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:10.095618 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-460f8629-de58-4aed-9460-55d81a0f1787 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:26:10.097116 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-460f8629-de58-4aed-9460-55d81a0f1787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:26:10.126708 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:10.175794 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 230/437] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:26:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 22:26:10.177011 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d48d821-2bb7-46a8-93b7-55964afa73df None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:10.177304 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d48d821-2bb7-46a8-93b7-55964afa73df None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:10.177587 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d48d821-2bb7-46a8-93b7-55964afa73df None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:10.183308 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 231/438] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:26:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:26:10.184466 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 208/439] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Jul 20 22:26:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 22:26:10.188357 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1673eb-e60f-49d3-9b2c-dc857b411d51 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:10.188736 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1673eb-e60f-49d3-9b2c-dc857b411d51 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:10.189092 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1673eb-e60f-49d3-9b2c-dc857b411d51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:10.192372 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402424c5-37cb-4338-8e29-a6e7e268f827 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:10.206032 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-2a1673eb-e60f-49d3-9b2c-dc857b411d51 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:26:10.207244 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-2a1673eb-e60f-49d3-9b2c-dc857b411d51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:26:10.273218 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 232/440] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:26:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 22:26:10.274027 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1673eb-e60f-49d3-9b2c-dc857b411d51 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:10.289080 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402424c5-37cb-4338-8e29-a6e7e268f827 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:10.291099 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:10.324253 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7faa9d-3f34-464f-87b8-d66f842b8665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:10.325138 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7faa9d-3f34-464f-87b8-d66f842b8665 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:10.325425 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7faa9d-3f34-464f-87b8-d66f842b8665 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:10.325700 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7faa9d-3f34-464f-87b8-d66f842b8665 None admin] PATH_INFO: `/v3/projects/7165d638ee164d27903602ef884fcf03` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:10.326991 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7faa9d-3f34-464f-87b8-d66f842b8665 None admin] RBAC: Authorizing `identity:get_project(project_id=7165d638ee164d27903602ef884fcf03)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:10.328457 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7faa9d-3f34-464f-87b8-d66f842b8665 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:10.330139 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 209/441] 10.222.0.81 () {56 vars in 1186 bytes} [Wed Jul 20 22:26:10 2022] GET /identity/v3/projects/7165d638ee164d27903602ef884fcf03 => generated 302 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 22:26:10.330885 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7faa9d-3f34-464f-87b8-d66f842b8665 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:10.347607 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:10.377049 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1673eb-e60f-49d3-9b2c-dc857b411d51 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:10.410032 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925b5a7a-3ee8-4147-929f-8424ef9d4bed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:10.410847 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925b5a7a-3ee8-4147-929f-8424ef9d4bed None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:10.411160 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925b5a7a-3ee8-4147-929f-8424ef9d4bed None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:10.411476 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925b5a7a-3ee8-4147-929f-8424ef9d4bed None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:10.421751 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925b5a7a-3ee8-4147-929f-8424ef9d4bed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 22:26:10.427412 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7faa9d-3f34-464f-87b8-d66f842b8665 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:10.430913 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-925b5a7a-3ee8-4147-929f-8424ef9d4bed None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 22:26:10.430913 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:10.430913 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:10.430913 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:10.430913 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:10.430913 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:10.430913 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:10.430913 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:10.430913 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:10.430913 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:10.430913 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:10.430913 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:10.430913 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 22:26:10.430913 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 22:26:10.430913 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 22:26:10.430913 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 22:26:10.432017 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 22:26:10.432017 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 22:26:10.432017 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 22:26:10.432017 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 22:26:10.432017 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:10.432017 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:10.432017 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:10.432017 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:10.432017 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:10.432017 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:10.432017 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:10.432017 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:10.432017 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:10.432017 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:10.432017 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:10.432017 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:10.433196 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:10.433196 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:10.433196 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:10.433196 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:10.433196 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 22:26:10.433196 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 22:26:10.433196 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 22:26:10.433196 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 22:26:10.433196 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 22:26:10.433196 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 22:26:10.433196 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 22:26:10.433196 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:26:10.433196 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 233/442] 10.222.0.81 () {56 vars in 1114 bytes} [Wed Jul 20 22:26:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:10.434785 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925b5a7a-3ee8-4147-929f-8424ef9d4bed None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:10.463062 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e20438-6860-4f5a-8504-9d032bc50859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:10.463873 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e20438-6860-4f5a-8504-9d032bc50859 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:10.464166 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e20438-6860-4f5a-8504-9d032bc50859 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:10.464439 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e20438-6860-4f5a-8504-9d032bc50859 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:10.474520 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e20438-6860-4f5a-8504-9d032bc50859 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 22:26:10.483887 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-77e20438-6860-4f5a-8504-9d032bc50859 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 22:26:10.483887 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:10.483887 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:10.483887 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:10.483887 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:10.483887 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:10.483887 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:10.483887 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:10.483887 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:10.483887 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:10.483887 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:10.483887 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:10.483887 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 22:26:10.483887 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 22:26:10.483887 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 22:26:10.483887 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 22:26:10.484742 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 22:26:10.484742 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 22:26:10.484742 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 22:26:10.484742 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 22:26:10.484742 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:10.484742 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:10.484742 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:10.484742 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:10.484742 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:10.484742 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:10.484742 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:10.484742 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:10.484742 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:10.484742 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:10.484742 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:10.484742 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:10.485527 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:10.485527 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:10.485527 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:10.485527 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:10.485527 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 22:26:10.485527 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 22:26:10.485527 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 22:26:10.485527 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 22:26:10.485527 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 22:26:10.485527 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 22:26:10.485527 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 22:26:10.485527 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:26:10.485527 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 210/443] 10.222.0.81 () {56 vars in 1096 bytes} [Wed Jul 20 22:26:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:10.486177 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e20438-6860-4f5a-8504-9d032bc50859 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:10.520426 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:10.545401 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925b5a7a-3ee8-4147-929f-8424ef9d4bed None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:10.563683 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:10.580069 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd7c358-9740-4402-aad1-302bfd512bce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:10.580926 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd7c358-9740-4402-aad1-302bfd512bce None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:10.581212 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd7c358-9740-4402-aad1-302bfd512bce None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:10.581492 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd7c358-9740-4402-aad1-302bfd512bce None admin] PATH_INFO: `/v3/groups/f5b13eb0527d4f748d667e2b1d087e54` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:10.582690 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd7c358-9740-4402-aad1-302bfd512bce None admin] RBAC: Authorizing `identity:get_group(group_id=f5b13eb0527d4f748d667e2b1d087e54)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:10.584169 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd7c358-9740-4402-aad1-302bfd512bce None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:10.585810 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 234/444] 10.222.0.81 () {56 vars in 1180 bytes} [Wed Jul 20 22:26:10 2022] GET /identity/v3/groups/f5b13eb0527d4f748d667e2b1d087e54 => generated 226 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:10.586185 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e20438-6860-4f5a-8504-9d032bc50859 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:10.586570 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd7c358-9740-4402-aad1-302bfd512bce None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:10.619359 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04bc572-5d4e-41c0-9fbb-3346167f2d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:10.620034 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04bc572-5d4e-41c0-9fbb-3346167f2d2f None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:10.620317 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04bc572-5d4e-41c0-9fbb-3346167f2d2f None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:10.620590 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04bc572-5d4e-41c0-9fbb-3346167f2d2f None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:10.621327 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04bc572-5d4e-41c0-9fbb-3346167f2d2f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:10.621637 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04bc572-5d4e-41c0-9fbb-3346167f2d2f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:10.622991 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04bc572-5d4e-41c0-9fbb-3346167f2d2f None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:10.630966 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 211/445] 10.222.0.81 () {56 vars in 1111 bytes} [Wed Jul 20 22:26:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:10.632378 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04bc572-5d4e-41c0-9fbb-3346167f2d2f None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:10.689163 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd7c358-9740-4402-aad1-302bfd512bce None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:10.723430 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fca8da4-b1e8-433e-8999-7e872ae59647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:10.724086 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fca8da4-b1e8-433e-8999-7e872ae59647 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:10.724378 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fca8da4-b1e8-433e-8999-7e872ae59647 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:10.724670 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fca8da4-b1e8-433e-8999-7e872ae59647 None admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:10.725429 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fca8da4-b1e8-433e-8999-7e872ae59647 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:10.725805 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fca8da4-b1e8-433e-8999-7e872ae59647 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:10.727213 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fca8da4-b1e8-433e-8999-7e872ae59647 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:10.729141 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04bc572-5d4e-41c0-9fbb-3346167f2d2f None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:10.734954 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 235/446] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Jul 20 22:26:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:10.735748 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fca8da4-b1e8-433e-8999-7e872ae59647 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:10.746259 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:10.766641 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfd6414-7f0d-4ff5-9045-a5e282ef91cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:10.767397 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfd6414-7f0d-4ff5-9045-a5e282ef91cc None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:10.767687 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfd6414-7f0d-4ff5-9045-a5e282ef91cc None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:10.769519 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfd6414-7f0d-4ff5-9045-a5e282ef91cc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:10.769519 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfd6414-7f0d-4ff5-9045-a5e282ef91cc None admin] RBAC: Adding query filter params (group.id=f5b13eb0527d4f748d667e2b1d087e54, role.id=559942c4de1744169f8b5e0404ecde6b, scope.project.id=7165d638ee164d27903602ef884fcf03) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:10.769519 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfd6414-7f0d-4ff5-9045-a5e282ef91cc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:10.770576 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfd6414-7f0d-4ff5-9045-a5e282ef91cc None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:10.776881 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 212/447] 10.222.0.81 () {56 vars in 1376 bytes} [Wed Jul 20 22:26:10 2022] GET /identity/v3/role_assignments?group.id=f5b13eb0527d4f748d667e2b1d087e54&scope.project.id=7165d638ee164d27903602ef884fcf03&role.id=559942c4de1744169f8b5e0404ecde6b => generated 611 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:10.777656 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfd6414-7f0d-4ff5-9045-a5e282ef91cc None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:10.780343 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:10.848193 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fca8da4-b1e8-433e-8999-7e872ae59647 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:10.886614 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ce41df-6a54-46a3-b0a8-2515047971e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:10.887697 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ce41df-6a54-46a3-b0a8-2515047971e2 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:10.887992 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ce41df-6a54-46a3-b0a8-2515047971e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:10.888276 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ce41df-6a54-46a3-b0a8-2515047971e2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:10.900325 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfd6414-7f0d-4ff5-9045-a5e282ef91cc None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:10.942531 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205878bc-89d6-490d-aa17-50050d4264d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:10.943161 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205878bc-89d6-490d-aa17-50050d4264d2 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:10.943456 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205878bc-89d6-490d-aa17-50050d4264d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:10.943813 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205878bc-89d6-490d-aa17-50050d4264d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:10.968431 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ce41df-6a54-46a3-b0a8-2515047971e2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:10.969987 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ce41df-6a54-46a3-b0a8-2515047971e2 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:10.980849 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:11.030866 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205878bc-89d6-490d-aa17-50050d4264d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:11.032388 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205878bc-89d6-490d-aa17-50050d4264d2 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:11.035565 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:11.094027 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 236/448] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Jul 20 22:26:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 22:26:11.094929 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ce41df-6a54-46a3-b0a8-2515047971e2 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:11.155879 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 213/449] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Jul 20 22:26:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 22:26:11.161550 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205878bc-89d6-490d-aa17-50050d4264d2 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:11.206195 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ce41df-6a54-46a3-b0a8-2515047971e2 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:11.216167 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:11.241364 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335e2fde-13cf-45ca-a4c7-0a86209c056d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:11.242247 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335e2fde-13cf-45ca-a4c7-0a86209c056d None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:11.242537 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335e2fde-13cf-45ca-a4c7-0a86209c056d None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:11.242814 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335e2fde-13cf-45ca-a4c7-0a86209c056d None admin] PATH_INFO: `/v3/projects/7165d638ee164d27903602ef884fcf03` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:11.244021 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335e2fde-13cf-45ca-a4c7-0a86209c056d None admin] RBAC: Authorizing `identity:get_project(project_id=7165d638ee164d27903602ef884fcf03)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:11.245472 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335e2fde-13cf-45ca-a4c7-0a86209c056d None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:11.247186 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 237/450] 10.222.0.81 () {56 vars in 1186 bytes} [Wed Jul 20 22:26:11 2022] GET /identity/v3/projects/7165d638ee164d27903602ef884fcf03 => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:11.252568 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335e2fde-13cf-45ca-a4c7-0a86209c056d None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:11.261366 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205878bc-89d6-490d-aa17-50050d4264d2 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:11.262907 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:11.294236 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc40d8f-51ff-4d14-9d44-e6abf29cb042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:11.295148 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc40d8f-51ff-4d14-9d44-e6abf29cb042 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:11.295434 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc40d8f-51ff-4d14-9d44-e6abf29cb042 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:11.295723 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc40d8f-51ff-4d14-9d44-e6abf29cb042 None admin] PATH_INFO: `/v3/projects/015b89c7582349b2a89f8bbaea747017` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:11.296923 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc40d8f-51ff-4d14-9d44-e6abf29cb042 None admin] RBAC: Authorizing `identity:get_project(project_id=015b89c7582349b2a89f8bbaea747017)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:11.298659 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc40d8f-51ff-4d14-9d44-e6abf29cb042 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:11.303290 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 214/451] 10.222.0.81 () {56 vars in 1186 bytes} [Wed Jul 20 22:26:11 2022] GET /identity/v3/projects/015b89c7582349b2a89f8bbaea747017 => generated 344 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 22:26:11.306457 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc40d8f-51ff-4d14-9d44-e6abf29cb042 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:11.352061 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335e2fde-13cf-45ca-a4c7-0a86209c056d None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:11.386418 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce95ef91-8db4-49ff-80ec-6f56dabe4839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:11.387300 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce95ef91-8db4-49ff-80ec-6f56dabe4839 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:11.387584 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce95ef91-8db4-49ff-80ec-6f56dabe4839 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:11.387869 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce95ef91-8db4-49ff-80ec-6f56dabe4839 None admin] PATH_INFO: `/v3/groups/f5b13eb0527d4f748d667e2b1d087e54` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:11.389049 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce95ef91-8db4-49ff-80ec-6f56dabe4839 None admin] RBAC: Authorizing `identity:get_group(group_id=f5b13eb0527d4f748d667e2b1d087e54)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:11.390493 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce95ef91-8db4-49ff-80ec-6f56dabe4839 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:11.392121 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 238/452] 10.222.0.81 () {56 vars in 1180 bytes} [Wed Jul 20 22:26:11 2022] GET /identity/v3/groups/f5b13eb0527d4f748d667e2b1d087e54 => generated 226 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 22:26:11.397866 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce95ef91-8db4-49ff-80ec-6f56dabe4839 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:11.402183 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc40d8f-51ff-4d14-9d44-e6abf29cb042 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:11.436796 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905db38a-b92c-4db6-8437-4089de41a397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:11.437726 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905db38a-b92c-4db6-8437-4089de41a397 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:11.438015 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905db38a-b92c-4db6-8437-4089de41a397 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:11.438294 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905db38a-b92c-4db6-8437-4089de41a397 None admin] PATH_INFO: `/v3/groups/4610f6e120da448ab12a53899c5ee533` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:11.439509 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905db38a-b92c-4db6-8437-4089de41a397 None admin] RBAC: Authorizing `identity:get_group(group_id=4610f6e120da448ab12a53899c5ee533)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:11.440929 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905db38a-b92c-4db6-8437-4089de41a397 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:11.442694 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 215/453] 10.222.0.81 () {56 vars in 1180 bytes} [Wed Jul 20 22:26:11 2022] GET /identity/v3/groups/4610f6e120da448ab12a53899c5ee533 => generated 229 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 22:26:11.443110 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:11.448339 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905db38a-b92c-4db6-8437-4089de41a397 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:11.487886 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:11.498261 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce95ef91-8db4-49ff-80ec-6f56dabe4839 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:11.530779 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd799b4-0f13-4a9f-90a4-3562440819c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:11.531464 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd799b4-0f13-4a9f-90a4-3562440819c7 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:11.531750 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd799b4-0f13-4a9f-90a4-3562440819c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:11.532032 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd799b4-0f13-4a9f-90a4-3562440819c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:11.532858 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd799b4-0f13-4a9f-90a4-3562440819c7 None admin] RBAC: Adding query filter params (group.id=f5b13eb0527d4f748d667e2b1d087e54, role.id=6d5a3e6a22fa4f4fa47afcfce0246a1f, scope.project.id=7165d638ee164d27903602ef884fcf03) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:11.533172 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd799b4-0f13-4a9f-90a4-3562440819c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:11.534603 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd799b4-0f13-4a9f-90a4-3562440819c7 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:11.540924 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 239/454] 10.222.0.81 () {56 vars in 1376 bytes} [Wed Jul 20 22:26:11 2022] GET /identity/v3/role_assignments?group.id=f5b13eb0527d4f748d667e2b1d087e54&scope.project.id=7165d638ee164d27903602ef884fcf03&role.id=6d5a3e6a22fa4f4fa47afcfce0246a1f => generated 611 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 22:26:11.547161 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905db38a-b92c-4db6-8437-4089de41a397 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:11.584463 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fca3966-2897-4133-bd1c-7bc138fbc018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:11.585717 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fca3966-2897-4133-bd1c-7bc138fbc018 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:11.586007 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fca3966-2897-4133-bd1c-7bc138fbc018 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:11.586289 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fca3966-2897-4133-bd1c-7bc138fbc018 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:11.587178 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fca3966-2897-4133-bd1c-7bc138fbc018 None admin] RBAC: Adding query filter params (group.id=4610f6e120da448ab12a53899c5ee533, role.id=d89b6c92d35a401a8517f1d47f6be843, scope.project.id=015b89c7582349b2a89f8bbaea747017) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:11.587501 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fca3966-2897-4133-bd1c-7bc138fbc018 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:11.588991 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fca3966-2897-4133-bd1c-7bc138fbc018 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:11.595766 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 216/455] 10.222.0.81 () {56 vars in 1376 bytes} [Wed Jul 20 22:26:11 2022] GET /identity/v3/role_assignments?group.id=4610f6e120da448ab12a53899c5ee533&scope.project.id=015b89c7582349b2a89f8bbaea747017&role.id=d89b6c92d35a401a8517f1d47f6be843 => generated 611 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 22:26:14.143283 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24542ee2-85cf-41eb-bfa3-9fc7eea17612 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:14.144313 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24542ee2-85cf-41eb-bfa3-9fc7eea17612 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:14.144811 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24542ee2-85cf-41eb-bfa3-9fc7eea17612 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:14.146608 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 240/456] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:26:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:26:14.152808 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5257ab-97d4-4f7f-aa03-f24c4a269ba9 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:14.153690 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5257ab-97d4-4f7f-aa03-f24c4a269ba9 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:14.154196 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5257ab-97d4-4f7f-aa03-f24c4a269ba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:14.179547 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-af5257ab-97d4-4f7f-aa03-f24c4a269ba9 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:26:14.181134 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-af5257ab-97d4-4f7f-aa03-f24c4a269ba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:26:14.268677 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 217/457] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:26:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 22:26:14.314032 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24542ee2-85cf-41eb-bfa3-9fc7eea17612 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:14.389862 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af9533d-628e-488b-8401-9e8e1abd0c9f None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:14.390491 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af9533d-628e-488b-8401-9e8e1abd0c9f None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:14.390971 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af9533d-628e-488b-8401-9e8e1abd0c9f None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:14.392838 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 218/458] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:26:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 22:26:14.398137 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336f914c-4127-453e-b89a-4154cb5250be None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:14.398677 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336f914c-4127-453e-b89a-4154cb5250be None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:14.399186 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336f914c-4127-453e-b89a-4154cb5250be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:14.427157 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-336f914c-4127-453e-b89a-4154cb5250be None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:26:14.428995 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-336f914c-4127-453e-b89a-4154cb5250be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:26:14.470469 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24542ee2-85cf-41eb-bfa3-9fc7eea17612 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:14.470986 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:14.489819 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:14.520644 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70e6330-9b8c-40a3-a7a9-3d6db6babe0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:14.521456 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70e6330-9b8c-40a3-a7a9-3d6db6babe0f None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:14.521943 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70e6330-9b8c-40a3-a7a9-3d6db6babe0f None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:14.522228 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70e6330-9b8c-40a3-a7a9-3d6db6babe0f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:14.535047 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70e6330-9b8c-40a3-a7a9-3d6db6babe0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:14.536525 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70e6330-9b8c-40a3-a7a9-3d6db6babe0f None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:14.536723 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 219/459] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:26:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 22:26:14.556495 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-a70e6330-9b8c-40a3-a7a9-3d6db6babe0f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:26:14.556495 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:14.556495 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:14.556495 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:14.556495 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:14.556495 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:14.556495 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:14.556495 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:14.556495 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:14.556495 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:14.556495 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:14.556495 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:14.556495 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 22:26:14.556495 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 22:26:14.556495 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 22:26:14.557394 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 22:26:14.557394 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:14.557394 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:14.557394 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:14.557394 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:14.557394 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:14.557394 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:14.557394 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:14.557394 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:14.557394 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:14.557394 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:14.557394 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:14.557394 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:14.557394 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:14.557394 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:14.557394 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:14.557394 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:14.558235 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 22:26:14.558235 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 22:26:14.558235 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:26:14.558235 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:26:14.558235 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 241/460] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Jul 20 22:26:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:14.563642 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336f914c-4127-453e-b89a-4154cb5250be None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:14.568653 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70e6330-9b8c-40a3-a7a9-3d6db6babe0f None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:14.674478 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70e6330-9b8c-40a3-a7a9-3d6db6babe0f None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:14.711547 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b88c131-9927-4b9e-8ca8-047eb559d008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:14.712759 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b88c131-9927-4b9e-8ca8-047eb559d008 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:14.713047 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b88c131-9927-4b9e-8ca8-047eb559d008 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:14.713328 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b88c131-9927-4b9e-8ca8-047eb559d008 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:14.724747 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b88c131-9927-4b9e-8ca8-047eb559d008 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:14.726346 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b88c131-9927-4b9e-8ca8-047eb559d008 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:14.746702 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-2b88c131-9927-4b9e-8ca8-047eb559d008 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:26:14.746702 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:14.746702 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:14.746702 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:14.746702 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:14.746702 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:14.746702 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:14.746702 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:14.746702 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:14.746702 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:14.746702 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:14.746702 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:14.746702 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 22:26:14.746702 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 22:26:14.746702 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 22:26:14.749281 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 22:26:14.749281 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:14.749281 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:14.749281 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:14.749281 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:14.749281 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:14.749281 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:14.749281 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:14.749281 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:14.749281 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:14.749281 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:14.749281 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:14.749281 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:14.749281 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:14.749281 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:14.749281 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:14.749281 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:14.752041 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 22:26:14.752041 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 22:26:14.752041 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:26:14.752041 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:26:14.755039 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 242/461] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Jul 20 22:26:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:14.755039 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b88c131-9927-4b9e-8ca8-047eb559d008 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:14.779391 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:14.840245 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336f914c-4127-453e-b89a-4154cb5250be None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:14.877731 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8bcb20-de3f-4a55-be92-c296894978c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:14.878411 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8bcb20-de3f-4a55-be92-c296894978c8 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:14.878702 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8bcb20-de3f-4a55-be92-c296894978c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:14.878982 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8bcb20-de3f-4a55-be92-c296894978c8 None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:14.879758 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8bcb20-de3f-4a55-be92-c296894978c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:14.880071 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8bcb20-de3f-4a55-be92-c296894978c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:14.880246 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b88c131-9927-4b9e-8ca8-047eb559d008 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:14.882034 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8bcb20-de3f-4a55-be92-c296894978c8 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:14.894047 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 220/462] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Jul 20 22:26:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:14.895289 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad2bfd8-e400-41fa-8481-1ce481246da3 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:14.895589 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad2bfd8-e400-41fa-8481-1ce481246da3 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:14.895870 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad2bfd8-e400-41fa-8481-1ce481246da3 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:14.897405 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 221/463] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:26:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:26:14.898504 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d96675-afc9-48af-ac9b-740f124c83ad None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:14.898841 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d96675-afc9-48af-ac9b-740f124c83ad None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:14.899168 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d96675-afc9-48af-ac9b-740f124c83ad None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:14.900383 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 222/464] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:26:14 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:26:14.905671 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf4b32f-78cf-403e-820e-da5c83a416ad None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:14.905970 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf4b32f-78cf-403e-820e-da5c83a416ad None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:14.906256 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf4b32f-78cf-403e-820e-da5c83a416ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:14.922068 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad29385-85c8-4bfc-9e8e-af8d85a73d64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:14.922875 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad29385-85c8-4bfc-9e8e-af8d85a73d64 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:14.923104 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-eaf4b32f-78cf-403e-820e-da5c83a416ad None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:26:14.923263 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad29385-85c8-4bfc-9e8e-af8d85a73d64 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:14.923424 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad29385-85c8-4bfc-9e8e-af8d85a73d64 None admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:14.924119 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad29385-85c8-4bfc-9e8e-af8d85a73d64 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:14.924284 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-eaf4b32f-78cf-403e-820e-da5c83a416ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:26:14.924429 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad29385-85c8-4bfc-9e8e-af8d85a73d64 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:14.925846 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad29385-85c8-4bfc-9e8e-af8d85a73d64 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:14.941893 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:14.946276 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 243/465] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Jul 20 22:26:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:14.950816 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5266bc5-7282-4f17-9651-b87fd9a234d8 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:14.950816 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5266bc5-7282-4f17-9651-b87fd9a234d8 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:14.950816 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5266bc5-7282-4f17-9651-b87fd9a234d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:14.967133 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-a5266bc5-7282-4f17-9651-b87fd9a234d8 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:26:14.968398 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-a5266bc5-7282-4f17-9651-b87fd9a234d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:26:15.007123 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 223/466] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:26:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 22:26:15.007123 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf4b32f-78cf-403e-820e-da5c83a416ad None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:15.057366 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 244/467] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:26:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 22:26:15.058027 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5266bc5-7282-4f17-9651-b87fd9a234d8 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:15.069957 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:15.121223 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf4b32f-78cf-403e-820e-da5c83a416ad None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:15.157786 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3f2126-8666-40de-8930-a186188f5ade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:15.158583 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3f2126-8666-40de-8930-a186188f5ade None admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:15.158869 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3f2126-8666-40de-8930-a186188f5ade None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:15.159176 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3f2126-8666-40de-8930-a186188f5ade None admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:15.159996 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3f2126-8666-40de-8930-a186188f5ade None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:15.161433 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3f2126-8666-40de-8930-a186188f5ade None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:15.165751 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5266bc5-7282-4f17-9651-b87fd9a234d8 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:15.202207 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b7fda3-04aa-43af-8e8c-727f766e9f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:15.207947 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b7fda3-04aa-43af-8e8c-727f766e9f92 None admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:15.207947 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b7fda3-04aa-43af-8e8c-727f766e9f92 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:15.207947 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b7fda3-04aa-43af-8e8c-727f766e9f92 None admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:15.207947 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b7fda3-04aa-43af-8e8c-727f766e9f92 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:15.207947 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b7fda3-04aa-43af-8e8c-727f766e9f92 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:15.241190 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 224/468] 10.222.0.81 () {60 vars in 1132 bytes} [Wed Jul 20 22:26:14 2022] POST /identity/v3/users => generated 246 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:26:15.242250 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3f2126-8666-40de-8930-a186188f5ade None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:15.255091 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:15.288154 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 245/469] 10.222.0.81 () {60 vars in 1132 bytes} [Wed Jul 20 22:26:15 2022] POST /identity/v3/users => generated 248 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:26:15.289882 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b7fda3-04aa-43af-8e8c-727f766e9f92 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:15.377419 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:15.434967 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b7fda3-04aa-43af-8e8c-727f766e9f92 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:15.444742 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3f2126-8666-40de-8930-a186188f5ade None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:15.488241 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c916f54b-1a1a-4c66-a530-a832fd1de9ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:15.488681 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c916f54b-1a1a-4c66-a530-a832fd1de9ce None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:15.489032 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c916f54b-1a1a-4c66-a530-a832fd1de9ce None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:15.489378 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c916f54b-1a1a-4c66-a530-a832fd1de9ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:15.490329 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cd5bdd-b074-40c4-9470-c258bd92a6dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:15.491951 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cd5bdd-b074-40c4-9470-c258bd92a6dc None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:15.492774 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cd5bdd-b074-40c4-9470-c258bd92a6dc None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:15.493239 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cd5bdd-b074-40c4-9470-c258bd92a6dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:15.502810 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c916f54b-1a1a-4c66-a530-a832fd1de9ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:15.504536 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c916f54b-1a1a-4c66-a530-a832fd1de9ce None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:15.509376 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cd5bdd-b074-40c4-9470-c258bd92a6dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:15.511094 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cd5bdd-b074-40c4-9470-c258bd92a6dc None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:15.525377 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-c916f54b-1a1a-4c66-a530-a832fd1de9ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:26:15.525377 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:15.525377 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:15.525377 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:15.525377 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:15.525377 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:15.525377 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:15.525377 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:15.525377 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:15.525377 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:15.525377 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:15.525377 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:15.525377 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 22:26:15.525377 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 22:26:15.525377 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 22:26:15.526500 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 22:26:15.526500 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:15.526500 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:15.526500 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:15.526500 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:15.526500 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:15.526500 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:15.526500 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:15.526500 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:15.526500 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:15.526500 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:15.526500 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:15.526500 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:15.526500 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:15.526500 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:15.526500 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:15.526500 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:15.527568 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 22:26:15.527568 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 22:26:15.527568 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:26:15.527568 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:26:15.533607 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 225/470] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Jul 20 22:26:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:15.535798 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c916f54b-1a1a-4c66-a530-a832fd1de9ce None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:15.538343 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-f8cd5bdd-b074-40c4-9470-c258bd92a6dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:26:15.538343 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:15.538343 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:15.538343 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:15.538343 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:15.538343 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:15.538343 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:15.538343 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:15.538343 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:15.538343 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:15.538343 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:15.538343 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:15.538343 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 22:26:15.538343 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 22:26:15.538343 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 22:26:15.539238 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 22:26:15.539238 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:15.539238 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:15.539238 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:15.539238 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:15.539238 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:15.539238 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:15.539238 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:15.539238 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:15.539238 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:15.539238 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:15.539238 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:15.539238 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:15.539238 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:15.539238 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:15.539238 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:15.539238 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:15.540472 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 22:26:15.540472 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 22:26:15.540472 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:26:15.540472 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:26:15.540472 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 246/471] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Jul 20 22:26:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:15.546465 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cd5bdd-b074-40c4-9470-c258bd92a6dc None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:15.610173 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:15.670792 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c916f54b-1a1a-4c66-a530-a832fd1de9ce None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:15.671795 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cd5bdd-b074-40c4-9470-c258bd92a6dc None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:15.683725 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:15.708884 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766987be-1c05-479b-893e-2a8613959bff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:15.710314 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766987be-1c05-479b-893e-2a8613959bff None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:15.710765 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766987be-1c05-479b-893e-2a8613959bff None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:15.711241 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766987be-1c05-479b-893e-2a8613959bff None admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:15.712289 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766987be-1c05-479b-893e-2a8613959bff None admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:15.712744 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766987be-1c05-479b-893e-2a8613959bff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:15.713058 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605a0c7a-1494-43b8-817b-8234323589cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:15.713895 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605a0c7a-1494-43b8-817b-8234323589cb None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:15.714256 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605a0c7a-1494-43b8-817b-8234323589cb None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:15.714605 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605a0c7a-1494-43b8-817b-8234323589cb None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:15.714958 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766987be-1c05-479b-893e-2a8613959bff None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:15.715755 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605a0c7a-1494-43b8-817b-8234323589cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:15.716134 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605a0c7a-1494-43b8-817b-8234323589cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:15.718313 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605a0c7a-1494-43b8-817b-8234323589cb None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:15.731729 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 247/472] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Jul 20 22:26:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:15.737183 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 226/473] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Jul 20 22:26:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:15.737309 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921b2623-244f-4337-ab0f-2105d5720fc0 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:15.737309 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921b2623-244f-4337-ab0f-2105d5720fc0 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:15.737309 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921b2623-244f-4337-ab0f-2105d5720fc0 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:15.738189 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605a0c7a-1494-43b8-817b-8234323589cb None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:15.739089 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 248/474] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:26:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:26:15.740134 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921b2623-244f-4337-ab0f-2105d5720fc0 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:15.859846 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605a0c7a-1494-43b8-817b-8234323589cb None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:15.886234 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921b2623-244f-4337-ab0f-2105d5720fc0 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:15.887045 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:15.906076 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8373f8a5-6b1c-49e7-8024-fad9860917a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:15.906768 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8373f8a5-6b1c-49e7-8024-fad9860917a0 None admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:15.907076 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8373f8a5-6b1c-49e7-8024-fad9860917a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:15.907360 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8373f8a5-6b1c-49e7-8024-fad9860917a0 None admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:15.908259 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8373f8a5-6b1c-49e7-8024-fad9860917a0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:15.910812 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8373f8a5-6b1c-49e7-8024-fad9860917a0 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:15.927349 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa79565-255e-45d8-b777-2f75315a7d73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:15.928581 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa79565-255e-45d8-b777-2f75315a7d73 None admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:15.933238 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa79565-255e-45d8-b777-2f75315a7d73 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:15.933238 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa79565-255e-45d8-b777-2f75315a7d73 None admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:15.933238 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa79565-255e-45d8-b777-2f75315a7d73 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:15.933238 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa79565-255e-45d8-b777-2f75315a7d73 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:15.984402 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 227/475] 10.222.0.81 () {60 vars in 1132 bytes} [Wed Jul 20 22:26:15 2022] POST /identity/v3/users => generated 249 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:26:15.985619 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4adf35b-b882-44f9-bbb0-415db814dc1c None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:15.985914 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4adf35b-b882-44f9-bbb0-415db814dc1c None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:15.986199 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4adf35b-b882-44f9-bbb0-415db814dc1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:16.007523 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 249/476] 10.222.0.81 () {60 vars in 1132 bytes} [Wed Jul 20 22:26:15 2022] POST /identity/v3/users => generated 248 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:26:16.014613 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-c4adf35b-b882-44f9-bbb0-415db814dc1c None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:26:16.015995 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-c4adf35b-b882-44f9-bbb0-415db814dc1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:26:16.114063 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 228/477] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:26:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 22:26:16.149847 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa79565-255e-45d8-b777-2f75315a7d73 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:16.196578 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:16.283831 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa79565-255e-45d8-b777-2f75315a7d73 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:16.320617 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116dbf95-bb70-48c6-a488-412e15cebd2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:16.321291 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116dbf95-bb70-48c6-a488-412e15cebd2e None admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:16.321576 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116dbf95-bb70-48c6-a488-412e15cebd2e None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:16.321915 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116dbf95-bb70-48c6-a488-412e15cebd2e None admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:16.322672 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116dbf95-bb70-48c6-a488-412e15cebd2e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:16.324106 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116dbf95-bb70-48c6-a488-412e15cebd2e None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:16.372293 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.sql.core [None req-116dbf95-bb70-48c6-a488-412e15cebd2e None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 20 22:26:16.372293 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 20 22:26:16.372293 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [parameters: {'id': '3990c91a56df4179bd45dc9951dc10bb', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 20 22:26:16.372293 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80441) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 20 22:26:16.390467 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-116dbf95-bb70-48c6-a488-412e15cebd2e None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 20 22:26:16.390467 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:16.390467 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 20 22:26:16.390467 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 20 22:26:16.390467 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 20 22:26:16.390467 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 20 22:26:16.390467 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 20 22:26:16.390467 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application result = self._query(query) Jul 20 22:26:16.390467 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 20 22:26:16.390467 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application conn.query(q) Jul 20 22:26:16.390467 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 20 22:26:16.390467 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 20 22:26:16.390467 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 20 22:26:16.390467 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application result.read() Jul 20 22:26:16.390467 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 20 22:26:16.391685 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 20 22:26:16.391685 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 20 22:26:16.391685 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 20 22:26:16.391685 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 20 22:26:16.391685 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 20 22:26:16.391685 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 20 22:26:16.391685 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 20 22:26:16.391685 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 20 22:26:16.391685 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:26:16.391685 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 20 22:26:16.391685 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:26:16.391685 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:16.391685 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 20 22:26:16.391685 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 20 22:26:16.391685 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 20 22:26:16.391685 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return ref.to_dict() Jul 20 22:26:16.391685 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 20 22:26:16.391685 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application next(self.gen) Jul 20 22:26:16.392573 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 20 22:26:16.392573 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application yield resource Jul 20 22:26:16.392573 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 20 22:26:16.392573 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application next(self.gen) Jul 20 22:26:16.392573 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 20 22:26:16.392573 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application self.session.rollback() Jul 20 22:26:16.392573 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 20 22:26:16.392573 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application self.force_reraise() Jul 20 22:26:16.392573 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 20 22:26:16.392573 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise self.value Jul 20 22:26:16.392573 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 20 22:26:16.392573 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 20 22:26:16.392573 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 20 22:26:16.392573 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application session.commit() Jul 20 22:26:16.392573 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 20 22:26:16.392573 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 20 22:26:16.392573 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 20 22:26:16.393411 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application self._prepare_impl() Jul 20 22:26:16.393411 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 20 22:26:16.393411 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application self.session.flush() Jul 20 22:26:16.393411 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 20 22:26:16.393411 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application self._flush(objects) Jul 20 22:26:16.393411 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 20 22:26:16.393411 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 20 22:26:16.393411 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 20 22:26:16.393411 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application compat.raise_( Jul 20 22:26:16.393411 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 20 22:26:16.393411 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception Jul 20 22:26:16.393411 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 20 22:26:16.393411 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application flush_context.execute() Jul 20 22:26:16.393411 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 20 22:26:16.393411 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rec.execute(self) Jul 20 22:26:16.393411 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 20 22:26:16.393411 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 20 22:26:16.394300 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 20 22:26:16.394300 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 20 22:26:16.394300 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 20 22:26:16.394300 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 20 22:26:16.394300 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 20 22:26:16.394300 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 20 22:26:16.394300 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 20 22:26:16.394300 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 20 22:26:16.394300 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 20 22:26:16.394300 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 20 22:26:16.394300 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 20 22:26:16.394300 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 20 22:26:16.394300 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 20 22:26:16.394300 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 20 22:26:16.394300 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 20 22:26:16.394300 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception Jul 20 22:26:16.396187 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 20 22:26:16.396187 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 20 22:26:16.396187 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 20 22:26:16.396187 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 20 22:26:16.396187 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 20 22:26:16.396187 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application result = self._query(query) Jul 20 22:26:16.396187 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 20 22:26:16.396187 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application conn.query(q) Jul 20 22:26:16.396187 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 20 22:26:16.396187 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 20 22:26:16.396187 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 20 22:26:16.396187 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application result.read() Jul 20 22:26:16.396187 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 20 22:26:16.396187 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 20 22:26:16.396187 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 20 22:26:16.396187 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 20 22:26:16.398378 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 20 22:26:16.398378 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 20 22:26:16.398378 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 20 22:26:16.398378 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 20 22:26:16.398378 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 20 22:26:16.398378 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 20 22:26:16.398378 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [parameters: {'id': '3990c91a56df4179bd45dc9951dc10bb', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 20 22:26:16.398378 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 20 22:26:16.398378 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:26:16.398378 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 22:26:16.398378 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:26:16.398378 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:16.398378 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:16.398378 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:16.398378 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:16.398378 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:16.400452 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:16.400452 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:16.400452 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:16.400452 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:16.400452 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:16.400452 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:16.400452 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 20 22:26:16.400452 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 20 22:26:16.400452 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:16.400452 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:16.400452 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 20 22:26:16.400452 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 20 22:26:16.400452 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 20 22:26:16.400452 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 20 22:26:16.400452 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 20 22:26:16.400452 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:26:16.401725 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 250/478] 10.222.0.81 () {60 vars in 1131 bytes} [Wed Jul 20 22:26:16 2022] POST /identity/v3/roles => generated 144 bytes in 243 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:16.401804 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4adf35b-b882-44f9-bbb0-415db814dc1c None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:16.511907 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4adf35b-b882-44f9-bbb0-415db814dc1c None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:16.519674 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:16.554558 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f9b704-c5d1-4ce9-b5e4-5c95d39ee6c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:16.555379 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f9b704-c5d1-4ce9-b5e4-5c95d39ee6c3 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:16.555630 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f9b704-c5d1-4ce9-b5e4-5c95d39ee6c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:16.555910 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f9b704-c5d1-4ce9-b5e4-5c95d39ee6c3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:16.575706 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f9b704-c5d1-4ce9-b5e4-5c95d39ee6c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 22:26:16.587487 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-66f9b704-c5d1-4ce9-b5e4-5c95d39ee6c3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 22:26:16.587487 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:16.587487 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:16.587487 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:16.587487 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:16.587487 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:16.587487 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:16.587487 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:16.587487 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:16.587487 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:16.587487 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:16.587487 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:16.587487 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 22:26:16.587487 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 22:26:16.587487 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 22:26:16.587487 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 22:26:16.589454 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 22:26:16.589454 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 22:26:16.589454 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 22:26:16.589454 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 22:26:16.589454 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:16.589454 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:16.589454 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:16.589454 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:16.589454 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:16.589454 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:16.589454 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:16.589454 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:16.589454 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:16.589454 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:16.589454 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:16.589454 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:16.591475 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:16.591475 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:16.591475 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:16.591475 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:16.591475 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 22:26:16.591475 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 22:26:16.591475 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 22:26:16.591475 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 22:26:16.591475 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 22:26:16.591475 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 22:26:16.591475 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 22:26:16.591475 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:26:16.591475 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 229/479] 10.222.0.81 () {56 vars in 1114 bytes} [Wed Jul 20 22:26:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:16.594769 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116dbf95-bb70-48c6-a488-412e15cebd2e None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:16.697612 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:16.707777 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116dbf95-bb70-48c6-a488-412e15cebd2e None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:16.746414 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e401fabd-6a5b-47dd-9ede-985bc8239924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:16.747122 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e401fabd-6a5b-47dd-9ede-985bc8239924 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:16.747420 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e401fabd-6a5b-47dd-9ede-985bc8239924 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:16.747707 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e401fabd-6a5b-47dd-9ede-985bc8239924 None admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:16.748470 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e401fabd-6a5b-47dd-9ede-985bc8239924 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:16.748787 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e401fabd-6a5b-47dd-9ede-985bc8239924 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:16.750177 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e401fabd-6a5b-47dd-9ede-985bc8239924 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:16.762464 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 251/480] 10.222.0.81 () {56 vars in 1111 bytes} [Wed Jul 20 22:26:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:17.793771 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675fb5ff-124e-4dd1-a834-28061744d87c None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:17.794331 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675fb5ff-124e-4dd1-a834-28061744d87c None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:17.794331 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675fb5ff-124e-4dd1-a834-28061744d87c None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:17.795985 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 230/481] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:26:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:26:17.808604 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6881f1dc-e9b8-42fc-ab43-90a724f3fa7e None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:17.809062 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6881f1dc-e9b8-42fc-ab43-90a724f3fa7e None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:17.809244 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6881f1dc-e9b8-42fc-ab43-90a724f3fa7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:17.831287 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-6881f1dc-e9b8-42fc-ab43-90a724f3fa7e None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:26:17.832483 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-6881f1dc-e9b8-42fc-ab43-90a724f3fa7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:26:17.929750 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 252/482] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:26:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 22:26:17.937969 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46555adb-d6b2-44d8-847f-9fdf25a98095 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:17.938261 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46555adb-d6b2-44d8-847f-9fdf25a98095 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:17.938543 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46555adb-d6b2-44d8-847f-9fdf25a98095 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:17.940044 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 231/483] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:26:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:26:17.945646 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fdf5a9-c2c8-478f-873a-22611b81b13b None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:17.945940 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fdf5a9-c2c8-478f-873a-22611b81b13b None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:17.946225 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fdf5a9-c2c8-478f-873a-22611b81b13b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:17.963105 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46555adb-d6b2-44d8-847f-9fdf25a98095 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:17.965721 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-74fdf5a9-c2c8-478f-873a-22611b81b13b None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:26:17.966905 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-74fdf5a9-c2c8-478f-873a-22611b81b13b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:26:18.097540 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 253/484] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:26:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 22:26:18.103190 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46555adb-d6b2-44d8-847f-9fdf25a98095 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:18.148327 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fdf5a9-c2c8-478f-873a-22611b81b13b None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:18.156802 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aafda03-d569-4a8c-a6f5-11740fb32ab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:18.157638 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aafda03-d569-4a8c-a6f5-11740fb32ab1 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:18.157926 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aafda03-d569-4a8c-a6f5-11740fb32ab1 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:18.158206 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aafda03-d569-4a8c-a6f5-11740fb32ab1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:18.170305 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:18.174143 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aafda03-d569-4a8c-a6f5-11740fb32ab1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 22:26:18.183460 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-9aafda03-d569-4a8c-a6f5-11740fb32ab1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 22:26:18.183460 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:18.183460 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:18.183460 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:18.183460 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:18.183460 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:18.183460 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:18.183460 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:18.183460 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:18.183460 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:18.183460 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:18.183460 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:18.183460 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 22:26:18.183460 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 22:26:18.183460 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 22:26:18.183460 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 22:26:18.190516 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 22:26:18.190516 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 22:26:18.190516 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 22:26:18.190516 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 22:26:18.190516 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:18.190516 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:18.190516 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:18.190516 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:18.190516 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:18.190516 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:18.190516 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:18.190516 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:18.190516 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:18.190516 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:18.190516 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:18.190516 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:18.191342 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:18.191342 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:18.191342 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:18.191342 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:18.191342 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 22:26:18.191342 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 22:26:18.191342 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 22:26:18.191342 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 22:26:18.191342 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 22:26:18.191342 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 22:26:18.191342 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 22:26:18.191342 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:26:18.191342 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 232/485] 10.222.0.81 () {56 vars in 1102 bytes} [Wed Jul 20 22:26:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:18.195312 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aafda03-d569-4a8c-a6f5-11740fb32ab1 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:18.226391 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:18.317847 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fdf5a9-c2c8-478f-873a-22611b81b13b None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:18.324826 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aafda03-d569-4a8c-a6f5-11740fb32ab1 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:18.355687 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3b0940-2536-41df-ac71-588ad32eb70e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:18.356916 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3b0940-2536-41df-ac71-588ad32eb70e None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:18.357422 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3b0940-2536-41df-ac71-588ad32eb70e None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:18.357953 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3b0940-2536-41df-ac71-588ad32eb70e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:18.377689 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c8fc58-7ca8-4f94-b79f-2ec03a0967db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:18.378735 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c8fc58-7ca8-4f94-b79f-2ec03a0967db None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:18.379256 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c8fc58-7ca8-4f94-b79f-2ec03a0967db None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:18.379754 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c8fc58-7ca8-4f94-b79f-2ec03a0967db None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:18.380272 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3b0940-2536-41df-ac71-588ad32eb70e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 22:26:18.381460 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c8fc58-7ca8-4f94-b79f-2ec03a0967db None admin] RBAC: Adding query filter params (name=service) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:18.381966 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c8fc58-7ca8-4f94-b79f-2ec03a0967db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:18.383606 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c8fc58-7ca8-4f94-b79f-2ec03a0967db None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:18.392828 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 233/486] 10.222.0.81 () {56 vars in 1103 bytes} [Wed Jul 20 22:26:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:18.399114 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c8fc58-7ca8-4f94-b79f-2ec03a0967db None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:18.400256 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-8f3b0940-2536-41df-ac71-588ad32eb70e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 22:26:18.400256 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:18.400256 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:18.400256 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:18.400256 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:18.400256 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:18.400256 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:18.400256 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:18.400256 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:18.400256 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:18.400256 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:18.400256 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:18.400256 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 22:26:18.400256 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 22:26:18.400256 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 22:26:18.400256 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 22:26:18.401132 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 22:26:18.401132 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 22:26:18.401132 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 22:26:18.401132 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 22:26:18.401132 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:18.401132 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:18.401132 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:18.401132 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:18.401132 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:18.401132 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:18.401132 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:18.401132 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:18.401132 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:18.401132 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:18.401132 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:18.401132 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:18.401913 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:18.401913 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:18.401913 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:18.401913 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:18.401913 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 22:26:18.401913 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 22:26:18.401913 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 22:26:18.401913 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 22:26:18.401913 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 22:26:18.401913 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 22:26:18.401913 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 22:26:18.401913 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:26:18.406234 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 254/487] 10.222.0.81 () {56 vars in 1102 bytes} [Wed Jul 20 22:26:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:18.407828 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef0a137-229f-4079-943e-775eb190a007 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:18.408323 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef0a137-229f-4079-943e-775eb190a007 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:18.408805 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef0a137-229f-4079-943e-775eb190a007 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:18.410750 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 255/488] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:26:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:26:18.411754 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef0a137-229f-4079-943e-775eb190a007 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:18.503933 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:18.511414 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:18.519990 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c8fc58-7ca8-4f94-b79f-2ec03a0967db None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:18.541049 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef0a137-229f-4079-943e-775eb190a007 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:18.565021 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd23095a-3bf6-4ccc-8891-8911542642a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:18.565850 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd23095a-3bf6-4ccc-8891-8911542642a2 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:18.566143 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd23095a-3bf6-4ccc-8891-8911542642a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:18.566424 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd23095a-3bf6-4ccc-8891-8911542642a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:18.579785 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc50b34-235d-4d01-b18f-bdf3ca0a8e47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:18.579953 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd23095a-3bf6-4ccc-8891-8911542642a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:18.580863 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc50b34-235d-4d01-b18f-bdf3ca0a8e47 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:18.581291 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd23095a-3bf6-4ccc-8891-8911542642a2 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:18.581531 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc50b34-235d-4d01-b18f-bdf3ca0a8e47 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:18.581988 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc50b34-235d-4d01-b18f-bdf3ca0a8e47 None admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:18.582935 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc50b34-235d-4d01-b18f-bdf3ca0a8e47 None admin] RBAC: Adding query filter params (name=service) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:18.583676 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc50b34-235d-4d01-b18f-bdf3ca0a8e47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:18.585918 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc50b34-235d-4d01-b18f-bdf3ca0a8e47 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:18.595438 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 256/489] 10.222.0.81 () {56 vars in 1103 bytes} [Wed Jul 20 22:26:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:18.597007 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57901310-d130-4709-89e1-a4c084e40774 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:18.597534 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57901310-d130-4709-89e1-a4c084e40774 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:18.598038 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57901310-d130-4709-89e1-a4c084e40774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:18.601264 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-fd23095a-3bf6-4ccc-8891-8911542642a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:26:18.601264 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:18.601264 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:18.601264 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:18.601264 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:18.601264 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:18.601264 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:18.601264 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:18.601264 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:18.601264 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:18.601264 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:18.601264 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:18.601264 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 22:26:18.601264 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 22:26:18.601264 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 22:26:18.602117 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 22:26:18.602117 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:18.602117 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:18.602117 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:18.602117 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:18.602117 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:18.602117 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:18.602117 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:18.602117 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:18.602117 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:18.602117 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:18.602117 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:18.602117 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:18.602117 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:18.602117 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:18.602117 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:18.602117 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:18.602957 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 22:26:18.602957 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 22:26:18.602957 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:26:18.602957 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:26:18.602957 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 234/490] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Jul 20 22:26:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:18.603687 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd23095a-3bf6-4ccc-8891-8911542642a2 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:18.620029 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-57901310-d130-4709-89e1-a4c084e40774 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:26:18.621694 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-57901310-d130-4709-89e1-a4c084e40774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:26:18.703289 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 257/491] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:26:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 22:26:18.704648 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57901310-d130-4709-89e1-a4c084e40774 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:18.722756 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd23095a-3bf6-4ccc-8891-8911542642a2 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:18.763025 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0a5ed1-688a-4483-91ce-96ac5c164b81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:18.763844 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0a5ed1-688a-4483-91ce-96ac5c164b81 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:18.764135 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0a5ed1-688a-4483-91ce-96ac5c164b81 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:18.764413 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0a5ed1-688a-4483-91ce-96ac5c164b81 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:18.776960 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:18.781059 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0a5ed1-688a-4483-91ce-96ac5c164b81 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:18.782520 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0a5ed1-688a-4483-91ce-96ac5c164b81 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:18.796190 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:18.808855 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-aa0a5ed1-688a-4483-91ce-96ac5c164b81 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:26:18.808855 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:18.808855 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:18.808855 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:18.808855 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:18.808855 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:18.808855 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:18.808855 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:18.808855 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:18.808855 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:18.808855 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:18.808855 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:18.808855 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 22:26:18.808855 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 22:26:18.808855 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 22:26:18.810083 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 22:26:18.810083 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:18.810083 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:18.810083 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:18.810083 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:18.810083 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:18.810083 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:18.810083 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:18.810083 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:18.810083 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:18.810083 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:18.810083 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:18.810083 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:18.810083 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:18.810083 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:18.810083 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:18.810083 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:18.810926 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 22:26:18.810926 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 22:26:18.810926 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:26:18.810926 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:26:18.812524 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 235/492] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Jul 20 22:26:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:18.817537 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0a5ed1-688a-4483-91ce-96ac5c164b81 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:18.849315 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57901310-d130-4709-89e1-a4c084e40774 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:18.888794 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e2cfe3-7abc-428f-89d5-6578552c08c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:18.889414 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e2cfe3-7abc-428f-89d5-6578552c08c9 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:18.889704 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e2cfe3-7abc-428f-89d5-6578552c08c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:18.889985 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e2cfe3-7abc-428f-89d5-6578552c08c9 None admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:18.890710 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e2cfe3-7abc-428f-89d5-6578552c08c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:18.892816 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e2cfe3-7abc-428f-89d5-6578552c08c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:18.894130 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e2cfe3-7abc-428f-89d5-6578552c08c9 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:18.906857 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 258/493] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Jul 20 22:26:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:18.907802 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e2cfe3-7abc-428f-89d5-6578552c08c9 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:18.933128 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0a5ed1-688a-4483-91ce-96ac5c164b81 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:18.970344 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277ab41b-9885-428d-9f6c-227727467db7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:18.971309 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277ab41b-9885-428d-9f6c-227727467db7 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:18.971599 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277ab41b-9885-428d-9f6c-227727467db7 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:18.971879 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277ab41b-9885-428d-9f6c-227727467db7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:18.983846 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277ab41b-9885-428d-9f6c-227727467db7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 22:26:18.992987 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-277ab41b-9885-428d-9f6c-227727467db7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 22:26:18.992987 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:18.992987 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:18.992987 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:18.992987 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:18.992987 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:18.992987 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:18.992987 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:18.992987 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:18.992987 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:18.992987 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:18.992987 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:18.992987 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 22:26:18.992987 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 22:26:18.992987 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 22:26:18.992987 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 22:26:18.993865 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 22:26:18.993865 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 22:26:18.993865 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 22:26:18.993865 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 22:26:18.993865 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:18.993865 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:18.993865 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:18.993865 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:18.993865 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:18.993865 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:18.993865 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:18.993865 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:18.993865 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:18.993865 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:18.993865 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:18.993865 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:18.994654 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:18.994654 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:18.994654 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:18.994654 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:18.994654 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 22:26:18.994654 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 22:26:18.994654 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 22:26:18.994654 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 22:26:18.994654 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 22:26:18.994654 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 22:26:18.994654 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 22:26:18.994654 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:26:18.994654 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 236/494] 10.222.0.81 () {56 vars in 1102 bytes} [Wed Jul 20 22:26:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:18.995895 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29278a77-fcc5-44da-a8e3-5fe3e8e8d6db None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:18.995895 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29278a77-fcc5-44da-a8e3-5fe3e8e8d6db None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:18.996178 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29278a77-fcc5-44da-a8e3-5fe3e8e8d6db None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:19.001623 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 237/495] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:26:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:26:19.001623 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29278a77-fcc5-44da-a8e3-5fe3e8e8d6db None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:19.023721 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e2cfe3-7abc-428f-89d5-6578552c08c9 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:19.050097 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:19.057397 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:19.062140 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02d6e4e-156c-4f28-aa10-affffcf46333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:19.062774 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02d6e4e-156c-4f28-aa10-affffcf46333 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:19.063053 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02d6e4e-156c-4f28-aa10-affffcf46333 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:19.063342 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02d6e4e-156c-4f28-aa10-affffcf46333 None admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:19.064126 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02d6e4e-156c-4f28-aa10-affffcf46333 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:19.064450 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02d6e4e-156c-4f28-aa10-affffcf46333 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:19.065843 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02d6e4e-156c-4f28-aa10-affffcf46333 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:19.083328 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 259/496] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Jul 20 22:26:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:19.083328 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02d6e4e-156c-4f28-aa10-affffcf46333 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:19.109322 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29278a77-fcc5-44da-a8e3-5fe3e8e8d6db None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:19.145423 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d722f11b-f280-4a21-ade1-24de85e375ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:19.146165 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d722f11b-f280-4a21-ade1-24de85e375ce None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:19.147036 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d722f11b-f280-4a21-ade1-24de85e375ce None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:19.147366 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d722f11b-f280-4a21-ade1-24de85e375ce None admin] PATH_INFO: `/v3/users/nova` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:19.155145 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d722f11b-f280-4a21-ade1-24de85e375ce None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:19.156610 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d722f11b-f280-4a21-ade1-24de85e375ce None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:19.173946 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-d722f11b-f280-4a21-ade1-24de85e375ce None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 20 22:26:19.173946 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:19.173946 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:19.173946 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:19.173946 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:19.173946 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:19.173946 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:19.173946 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:19.173946 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:19.173946 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:19.173946 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:19.173946 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:19.173946 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 22:26:19.173946 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 22:26:19.173946 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 22:26:19.173946 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 22:26:19.174895 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:19.174895 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:19.174895 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 22:26:19.174895 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 22:26:19.174895 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 22:26:19.174895 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 22:26:19.174895 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:19.174895 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:19.174895 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:19.174895 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:19.174895 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:19.174895 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:19.174895 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:19.174895 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:19.174895 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:19.174895 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:19.174895 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:19.174895 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:19.175826 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:19.175826 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:19.175826 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 22:26:19.175826 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 22:26:19.175826 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 22:26:19.175826 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 22:26:19.175826 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 22:26:19.175826 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 22:26:19.175826 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 20 22:26:19.175826 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:26:19.175826 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 238/497] 10.222.0.81 () {56 vars in 1093 bytes} [Wed Jul 20 22:26:18 2022] GET /identity/v3/users/nova => generated 82 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:19.176468 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493211fc-0b8f-4f58-8941-79354f1cb420 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:19.176534 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493211fc-0b8f-4f58-8941-79354f1cb420 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:19.177102 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493211fc-0b8f-4f58-8941-79354f1cb420 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:19.198885 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-493211fc-0b8f-4f58-8941-79354f1cb420 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:26:19.199443 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02d6e4e-156c-4f28-aa10-affffcf46333 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:19.200139 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-493211fc-0b8f-4f58-8941-79354f1cb420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:26:19.236371 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6c09a8-dc5a-4a43-81c8-ba76104e0850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:19.237178 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6c09a8-dc5a-4a43-81c8-ba76104e0850 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:19.237535 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6c09a8-dc5a-4a43-81c8-ba76104e0850 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:19.237900 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6c09a8-dc5a-4a43-81c8-ba76104e0850 None admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:19.238729 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6c09a8-dc5a-4a43-81c8-ba76104e0850 None admin] RBAC: Adding query filter params (name=service) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:19.239082 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6c09a8-dc5a-4a43-81c8-ba76104e0850 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:19.240569 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6c09a8-dc5a-4a43-81c8-ba76104e0850 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:19.249909 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 260/498] 10.222.0.81 () {56 vars in 1103 bytes} [Wed Jul 20 22:26:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:19.250715 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6c09a8-dc5a-4a43-81c8-ba76104e0850 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:19.287539 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 239/499] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:26:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 22:26:19.287539 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493211fc-0b8f-4f58-8941-79354f1cb420 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:19.311499 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:19.339143 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:19.368139 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6c09a8-dc5a-4a43-81c8-ba76104e0850 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:19.401653 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493211fc-0b8f-4f58-8941-79354f1cb420 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:19.405899 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfd027e-1d7b-419a-886f-cd7a7cb29dc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:19.406713 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfd027e-1d7b-419a-886f-cd7a7cb29dc0 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:19.407002 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfd027e-1d7b-419a-886f-cd7a7cb29dc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:19.407303 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfd027e-1d7b-419a-886f-cd7a7cb29dc0 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:19.415964 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfd027e-1d7b-419a-886f-cd7a7cb29dc0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:19.417414 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfd027e-1d7b-419a-886f-cd7a7cb29dc0 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:19.433120 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-9bfd027e-1d7b-419a-886f-cd7a7cb29dc0 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 20 22:26:19.433120 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:19.433120 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:19.433120 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:19.433120 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:19.433120 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:19.433120 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:19.433120 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:19.433120 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:19.433120 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:19.433120 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:19.433120 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:19.433120 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 22:26:19.433120 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 22:26:19.433120 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 22:26:19.433120 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 22:26:19.434044 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:19.434044 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:19.434044 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 22:26:19.434044 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 22:26:19.434044 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 22:26:19.434044 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 22:26:19.434044 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:19.434044 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:19.434044 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:19.434044 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:19.434044 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:19.434044 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:19.434044 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:19.434044 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:19.434044 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:19.434044 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:19.434044 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:19.434044 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:19.434940 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:19.434940 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:19.434940 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 22:26:19.434940 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 22:26:19.434940 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 22:26:19.434940 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 22:26:19.434940 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 22:26:19.434940 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 22:26:19.434940 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 20 22:26:19.434940 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:26:19.438976 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 261/500] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Jul 20 22:26:19 2022] GET /identity/v3/users/glance => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:19.439836 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfd027e-1d7b-419a-886f-cd7a7cb29dc0 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:19.440269 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ac424c-6203-4723-9a4c-d03dec372617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:19.441069 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ac424c-6203-4723-9a4c-d03dec372617 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:19.441354 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ac424c-6203-4723-9a4c-d03dec372617 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:19.441632 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ac424c-6203-4723-9a4c-d03dec372617 None admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:19.442621 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ac424c-6203-4723-9a4c-d03dec372617 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:19.442934 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ac424c-6203-4723-9a4c-d03dec372617 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:19.445302 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ac424c-6203-4723-9a4c-d03dec372617 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:19.489702 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 240/501] 10.222.0.81 () {56 vars in 1133 bytes} [Wed Jul 20 22:26:19 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:19.489702 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ac424c-6203-4723-9a4c-d03dec372617 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:19.547626 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfd027e-1d7b-419a-886f-cd7a7cb29dc0 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:19.554963 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:19.582232 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a7b70b-3336-4004-a260-999fc572cd30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:19.583057 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a7b70b-3336-4004-a260-999fc572cd30 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:19.583352 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a7b70b-3336-4004-a260-999fc572cd30 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:19.583635 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a7b70b-3336-4004-a260-999fc572cd30 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:19.584911 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:19.587854 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ac424c-6203-4723-9a4c-d03dec372617 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:19.593522 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a7b70b-3336-4004-a260-999fc572cd30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:19.595761 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a7b70b-3336-4004-a260-999fc572cd30 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:19.613812 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-98a7b70b-3336-4004-a260-999fc572cd30 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:26:19.613812 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:19.613812 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:19.613812 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:19.613812 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:19.613812 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:19.613812 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:19.613812 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:19.613812 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:19.613812 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:19.613812 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:19.613812 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:19.613812 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 22:26:19.613812 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 22:26:19.613812 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 22:26:19.614651 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 22:26:19.614651 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:19.614651 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:19.614651 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:19.614651 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:19.614651 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:19.614651 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:19.614651 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:19.614651 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:19.614651 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:19.614651 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:19.614651 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:19.614651 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:19.614651 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:19.614651 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:19.614651 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:19.614651 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:19.616063 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 22:26:19.616063 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 22:26:19.616063 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:26:19.616063 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:26:19.616063 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 262/502] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Jul 20 22:26:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:19.616327 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a7b70b-3336-4004-a260-999fc572cd30 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:19.625874 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a785fdc0-1b0f-4264-ae86-277f71ca2874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:19.626723 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a785fdc0-1b0f-4264-ae86-277f71ca2874 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:19.627036 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a785fdc0-1b0f-4264-ae86-277f71ca2874 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:19.627314 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a785fdc0-1b0f-4264-ae86-277f71ca2874 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:19.637944 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a785fdc0-1b0f-4264-ae86-277f71ca2874 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 22:26:19.646120 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-a785fdc0-1b0f-4264-ae86-277f71ca2874 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 22:26:19.646120 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:19.646120 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:19.646120 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:19.646120 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:19.646120 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:19.646120 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:19.646120 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:19.646120 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:19.646120 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:19.646120 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:19.646120 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:19.646120 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 22:26:19.646120 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 22:26:19.646120 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 22:26:19.646120 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 22:26:19.646990 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 22:26:19.646990 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 22:26:19.646990 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 22:26:19.646990 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 22:26:19.646990 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:19.646990 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:19.646990 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:19.646990 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:19.646990 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:19.646990 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:19.646990 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:19.646990 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:19.646990 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:19.646990 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:19.646990 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:19.646990 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:19.647815 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:19.647815 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:19.647815 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:19.647815 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:19.647815 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 22:26:19.647815 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 22:26:19.647815 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 22:26:19.647815 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 22:26:19.647815 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 22:26:19.647815 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 22:26:19.647815 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 22:26:19.647815 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:26:19.647815 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 241/503] 10.222.0.81 () {56 vars in 1102 bytes} [Wed Jul 20 22:26:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:19.648643 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a785fdc0-1b0f-4264-ae86-277f71ca2874 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:19.718637 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a7b70b-3336-4004-a260-999fc572cd30 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:19.750191 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a785fdc0-1b0f-4264-ae86-277f71ca2874 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:19.751847 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795a6af7-db47-4dae-a647-afee59441f7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:19.752516 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795a6af7-db47-4dae-a647-afee59441f7f None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:19.752799 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795a6af7-db47-4dae-a647-afee59441f7f None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:19.753077 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795a6af7-db47-4dae-a647-afee59441f7f None admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:19.753903 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795a6af7-db47-4dae-a647-afee59441f7f None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:19.754217 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795a6af7-db47-4dae-a647-afee59441f7f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:19.755719 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795a6af7-db47-4dae-a647-afee59441f7f None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:19.782118 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1996adc-26da-4815-a126-e42c8b5918cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:19.782765 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1996adc-26da-4815-a126-e42c8b5918cf None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:19.783076 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1996adc-26da-4815-a126-e42c8b5918cf None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:19.783353 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1996adc-26da-4815-a126-e42c8b5918cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:19.791792 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 263/504] 10.222.0.81 () {56 vars in 1137 bytes} [Wed Jul 20 22:26:19 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:19.793139 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c6a9cb-5962-468d-9789-34e525cd441c None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:19.793434 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c6a9cb-5962-468d-9789-34e525cd441c None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:19.793719 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c6a9cb-5962-468d-9789-34e525cd441c None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:19.794938 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 264/505] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:26:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:26:19.795652 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c6a9cb-5962-468d-9789-34e525cd441c None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:19.814247 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:19.815968 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:19.854197 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1996adc-26da-4815-a126-e42c8b5918cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:19.855713 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1996adc-26da-4815-a126-e42c8b5918cf None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:19.897671 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c6a9cb-5962-468d-9789-34e525cd441c None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:19.932252 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f706bd7d-3c40-4919-b5fd-eba3bf977d42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:19.932901 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f706bd7d-3c40-4919-b5fd-eba3bf977d42 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:19.933181 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f706bd7d-3c40-4919-b5fd-eba3bf977d42 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:19.933455 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f706bd7d-3c40-4919-b5fd-eba3bf977d42 None admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:19.934168 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f706bd7d-3c40-4919-b5fd-eba3bf977d42 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:19.934470 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f706bd7d-3c40-4919-b5fd-eba3bf977d42 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:19.935870 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f706bd7d-3c40-4919-b5fd-eba3bf977d42 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:19.947349 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 265/506] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Jul 20 22:26:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:19.948144 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f706bd7d-3c40-4919-b5fd-eba3bf977d42 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:19.960824 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 242/507] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Jul 20 22:26:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 22:26:19.961613 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1996adc-26da-4815-a126-e42c8b5918cf None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:20.019575 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:20.027760 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:20.049648 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f706bd7d-3c40-4919-b5fd-eba3bf977d42 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:20.059102 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1996adc-26da-4815-a126-e42c8b5918cf None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:20.083676 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12920e6-e35d-4ab3-b746-6479b7a8f5df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:20.084333 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12920e6-e35d-4ab3-b746-6479b7a8f5df None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:20.084618 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12920e6-e35d-4ab3-b746-6479b7a8f5df None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:20.084898 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12920e6-e35d-4ab3-b746-6479b7a8f5df None admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:20.085640 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12920e6-e35d-4ab3-b746-6479b7a8f5df None admin] RBAC: Adding query filter params (name=service) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:20.085962 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12920e6-e35d-4ab3-b746-6479b7a8f5df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:20.087401 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12920e6-e35d-4ab3-b746-6479b7a8f5df None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:20.093489 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c326dd31-683c-4496-9297-f61b4d29c4f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:20.094145 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c326dd31-683c-4496-9297-f61b4d29c4f8 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:20.094431 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c326dd31-683c-4496-9297-f61b4d29c4f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:20.094711 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c326dd31-683c-4496-9297-f61b4d29c4f8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:20.100410 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 266/508] 10.222.0.81 () {56 vars in 1103 bytes} [Wed Jul 20 22:26:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:20.100410 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb968fb-fd59-435e-83e6-b381fcd88e65 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:20.100410 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb968fb-fd59-435e-83e6-b381fcd88e65 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:20.100410 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb968fb-fd59-435e-83e6-b381fcd88e65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:20.112838 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-acb968fb-fd59-435e-83e6-b381fcd88e65 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:26:20.114031 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-acb968fb-fd59-435e-83e6-b381fcd88e65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:26:20.163991 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c326dd31-683c-4496-9297-f61b4d29c4f8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:20.165465 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c326dd31-683c-4496-9297-f61b4d29c4f8 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:20.178913 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 267/509] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:26:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 22:26:20.179870 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb968fb-fd59-435e-83e6-b381fcd88e65 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:20.238643 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:20.258877 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:20.279138 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 243/510] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Jul 20 22:26:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 22:26:20.279910 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c326dd31-683c-4496-9297-f61b4d29c4f8 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:20.292798 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb968fb-fd59-435e-83e6-b381fcd88e65 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:20.325872 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be92f158-b0ed-401e-8c44-41b2ec7bcccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:20.326700 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be92f158-b0ed-401e-8c44-41b2ec7bcccd None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:20.326982 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be92f158-b0ed-401e-8c44-41b2ec7bcccd None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:20.327287 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be92f158-b0ed-401e-8c44-41b2ec7bcccd None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:20.334980 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be92f158-b0ed-401e-8c44-41b2ec7bcccd None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:20.336490 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be92f158-b0ed-401e-8c44-41b2ec7bcccd None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:20.353453 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-be92f158-b0ed-401e-8c44-41b2ec7bcccd None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 20 22:26:20.353453 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:20.353453 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:20.353453 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:20.353453 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:20.353453 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:20.353453 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:20.353453 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:20.353453 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:20.353453 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:20.353453 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:20.353453 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:20.353453 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 22:26:20.353453 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 22:26:20.353453 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 22:26:20.353453 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 22:26:20.354563 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:20.354563 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:20.354563 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 22:26:20.354563 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 22:26:20.354563 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 22:26:20.354563 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 22:26:20.354563 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:20.354563 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:20.354563 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:20.354563 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:20.354563 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:20.354563 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:20.354563 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:20.354563 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:20.354563 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:20.354563 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:20.354563 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:20.354563 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:20.355511 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:20.355511 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:20.355511 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 22:26:20.355511 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 22:26:20.355511 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 22:26:20.355511 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 22:26:20.355511 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 22:26:20.355511 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 22:26:20.355511 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 20 22:26:20.355511 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:26:20.355511 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 268/511] 10.222.0.81 () {56 vars in 1102 bytes} [Wed Jul 20 22:26:20 2022] GET /identity/v3/users/neutron => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:20.356063 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be92f158-b0ed-401e-8c44-41b2ec7bcccd None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:20.381972 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c326dd31-683c-4496-9297-f61b4d29c4f8 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:20.416096 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba46904-9563-45df-afe9-8f3989ae74d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:20.416904 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba46904-9563-45df-afe9-8f3989ae74d4 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:20.417187 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba46904-9563-45df-afe9-8f3989ae74d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:20.417462 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba46904-9563-45df-afe9-8f3989ae74d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:20.427354 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba46904-9563-45df-afe9-8f3989ae74d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:20.428835 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba46904-9563-45df-afe9-8f3989ae74d4 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:20.447656 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-0ba46904-9563-45df-afe9-8f3989ae74d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:26:20.447656 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:20.447656 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:20.447656 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:20.447656 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:20.447656 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:20.447656 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:20.447656 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:20.447656 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:20.447656 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:20.447656 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:20.447656 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:20.447656 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 22:26:20.447656 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 22:26:20.447656 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 22:26:20.448668 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 22:26:20.448668 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:20.448668 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:20.448668 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:20.448668 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:20.448668 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:20.448668 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:20.448668 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:20.448668 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:20.448668 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:20.448668 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:20.448668 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:20.448668 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:20.448668 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:20.448668 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:20.448668 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:20.448668 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:20.449669 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 22:26:20.449669 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 22:26:20.449669 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:26:20.449669 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:26:20.449669 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 244/512] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Jul 20 22:26:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:20.449977 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba46904-9563-45df-afe9-8f3989ae74d4 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:20.460182 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be92f158-b0ed-401e-8c44-41b2ec7bcccd None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:20.466187 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:20.486221 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:20.493653 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb78700-bce1-4c41-b1a2-3326fdb0ac55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:20.494453 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb78700-bce1-4c41-b1a2-3326fdb0ac55 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:20.494737 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb78700-bce1-4c41-b1a2-3326fdb0ac55 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:20.495034 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb78700-bce1-4c41-b1a2-3326fdb0ac55 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:20.504958 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb78700-bce1-4c41-b1a2-3326fdb0ac55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:20.506959 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb78700-bce1-4c41-b1a2-3326fdb0ac55 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:20.525311 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-bcb78700-bce1-4c41-b1a2-3326fdb0ac55 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:26:20.525311 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:20.525311 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:20.525311 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:20.525311 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:20.525311 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:20.525311 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:20.525311 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:20.525311 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:20.525311 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:20.525311 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:20.525311 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:20.525311 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 22:26:20.525311 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 22:26:20.525311 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 22:26:20.526440 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 22:26:20.526440 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:20.526440 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:20.526440 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:20.526440 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:20.526440 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:20.526440 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:20.526440 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:20.526440 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:20.526440 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:20.526440 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:20.526440 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:20.526440 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:20.526440 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:20.526440 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:20.526440 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:20.526440 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:20.527743 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 22:26:20.527743 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 22:26:20.527743 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:26:20.527743 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:26:20.528009 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 269/513] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Jul 20 22:26:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:20.529236 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb78700-bce1-4c41-b1a2-3326fdb0ac55 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:20.557437 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba46904-9563-45df-afe9-8f3989ae74d4 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:20.591612 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adcef84-20fd-4a8e-a89c-788095c64fe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:20.592431 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adcef84-20fd-4a8e-a89c-788095c64fe9 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:20.592714 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adcef84-20fd-4a8e-a89c-788095c64fe9 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:20.592990 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adcef84-20fd-4a8e-a89c-788095c64fe9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:20.602992 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adcef84-20fd-4a8e-a89c-788095c64fe9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:20.604476 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adcef84-20fd-4a8e-a89c-788095c64fe9 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:20.622244 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-3adcef84-20fd-4a8e-a89c-788095c64fe9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:26:20.622244 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:20.622244 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:20.622244 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:20.622244 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:20.622244 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:20.622244 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:20.622244 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:20.622244 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:20.622244 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:20.622244 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:20.622244 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:20.622244 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 22:26:20.622244 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 22:26:20.622244 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 22:26:20.623343 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 22:26:20.623343 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:20.623343 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:20.623343 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:20.623343 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:20.623343 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:20.623343 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:20.623343 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:20.623343 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:20.623343 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:20.623343 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:20.623343 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:20.623343 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:20.623343 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:20.623343 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:20.623343 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:20.623343 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:20.624388 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 22:26:20.624388 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 22:26:20.624388 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:26:20.624388 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:26:20.628425 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 245/514] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Jul 20 22:26:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:20.629294 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adcef84-20fd-4a8e-a89c-788095c64fe9 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:20.637516 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb78700-bce1-4c41-b1a2-3326fdb0ac55 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:20.672131 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad090362-82ea-4e06-8537-4e00f44b174a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:20.672941 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad090362-82ea-4e06-8537-4e00f44b174a None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:20.673229 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad090362-82ea-4e06-8537-4e00f44b174a None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:20.673508 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad090362-82ea-4e06-8537-4e00f44b174a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:20.684248 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad090362-82ea-4e06-8537-4e00f44b174a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:20.685703 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad090362-82ea-4e06-8537-4e00f44b174a None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:20.696467 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:20.703999 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-ad090362-82ea-4e06-8537-4e00f44b174a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:26:20.703999 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:20.703999 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:20.703999 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:20.703999 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:20.703999 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:20.703999 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:20.703999 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:20.703999 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:20.703999 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:20.703999 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:20.703999 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:20.703999 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 22:26:20.703999 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 22:26:20.703999 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 22:26:20.714637 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 22:26:20.714637 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:20.714637 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:20.714637 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:20.714637 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:20.714637 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:20.714637 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:20.714637 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:20.714637 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:20.714637 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:20.714637 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:20.714637 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:20.714637 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:20.714637 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:20.714637 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:20.714637 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:20.714637 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:20.715683 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 22:26:20.715683 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 22:26:20.715683 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:26:20.715683 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:26:20.715683 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 270/515] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Jul 20 22:26:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:20.715683 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad090362-82ea-4e06-8537-4e00f44b174a None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:20.738529 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adcef84-20fd-4a8e-a89c-788095c64fe9 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:20.738702 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:20.773534 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1545fc2b-4514-468a-b97b-7184d16ab2c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:20.783280 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1545fc2b-4514-468a-b97b-7184d16ab2c6 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:20.783280 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1545fc2b-4514-468a-b97b-7184d16ab2c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:20.783280 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1545fc2b-4514-468a-b97b-7184d16ab2c6 None admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:20.783280 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1545fc2b-4514-468a-b97b-7184d16ab2c6 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:20.783280 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1545fc2b-4514-468a-b97b-7184d16ab2c6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:20.783280 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1545fc2b-4514-468a-b97b-7184d16ab2c6 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:20.795145 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 246/516] 10.222.0.81 () {56 vars in 1139 bytes} [Wed Jul 20 22:26:20 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:20.795977 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1545fc2b-4514-468a-b97b-7184d16ab2c6 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:20.808928 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad090362-82ea-4e06-8537-4e00f44b174a None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:20.844101 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072aa824-1ae1-4085-a729-15f31be6f693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:20.844767 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072aa824-1ae1-4085-a729-15f31be6f693 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:20.845058 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072aa824-1ae1-4085-a729-15f31be6f693 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:20.845343 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072aa824-1ae1-4085-a729-15f31be6f693 None admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:20.846067 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072aa824-1ae1-4085-a729-15f31be6f693 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:20.846380 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072aa824-1ae1-4085-a729-15f31be6f693 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:20.847804 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072aa824-1ae1-4085-a729-15f31be6f693 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:20.859722 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 271/517] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Jul 20 22:26:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:20.860538 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072aa824-1ae1-4085-a729-15f31be6f693 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:20.897559 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1545fc2b-4514-468a-b97b-7184d16ab2c6 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:20.921988 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:20.934814 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e546d1ee-16ef-469e-92d1-e45cd0f137ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:20.935552 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e546d1ee-16ef-469e-92d1-e45cd0f137ad None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:20.935842 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e546d1ee-16ef-469e-92d1-e45cd0f137ad None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:20.936122 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e546d1ee-16ef-469e-92d1-e45cd0f137ad None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:20.937939 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e546d1ee-16ef-469e-92d1-e45cd0f137ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:20.938253 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e546d1ee-16ef-469e-92d1-e45cd0f137ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:20.939660 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e546d1ee-16ef-469e-92d1-e45cd0f137ad None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:20.951901 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 247/518] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Jul 20 22:26:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:20.952703 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e546d1ee-16ef-469e-92d1-e45cd0f137ad None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:20.954100 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:20.962812 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072aa824-1ae1-4085-a729-15f31be6f693 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:20.999785 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001b3cd6-4b48-45b4-85c2-339268e396f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:21.000467 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001b3cd6-4b48-45b4-85c2-339268e396f7 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:21.000937 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001b3cd6-4b48-45b4-85c2-339268e396f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:21.001389 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001b3cd6-4b48-45b4-85c2-339268e396f7 None admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:21.002131 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001b3cd6-4b48-45b4-85c2-339268e396f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:21.002789 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001b3cd6-4b48-45b4-85c2-339268e396f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:21.004201 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001b3cd6-4b48-45b4-85c2-339268e396f7 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:21.015992 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 272/519] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Jul 20 22:26:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:21.016779 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001b3cd6-4b48-45b4-85c2-339268e396f7 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:21.055769 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e546d1ee-16ef-469e-92d1-e45cd0f137ad None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:21.090372 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90640e7e-f646-4c17-bcbe-d65f04ea70c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:21.091073 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90640e7e-f646-4c17-bcbe-d65f04ea70c7 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:21.091360 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90640e7e-f646-4c17-bcbe-d65f04ea70c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:21.091636 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90640e7e-f646-4c17-bcbe-d65f04ea70c7 None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:21.092355 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90640e7e-f646-4c17-bcbe-d65f04ea70c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:21.092662 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90640e7e-f646-4c17-bcbe-d65f04ea70c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:21.094023 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90640e7e-f646-4c17-bcbe-d65f04ea70c7 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:21.106102 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 248/520] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Jul 20 22:26:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:21.106898 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90640e7e-f646-4c17-bcbe-d65f04ea70c7 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:21.124770 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001b3cd6-4b48-45b4-85c2-339268e396f7 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:21.150218 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:21.160559 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbcb607-a779-4f42-82ea-2dcd369c717a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:21.161209 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbcb607-a779-4f42-82ea-2dcd369c717a None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:21.161520 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbcb607-a779-4f42-82ea-2dcd369c717a None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:21.161858 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbcb607-a779-4f42-82ea-2dcd369c717a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:21.184018 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:21.206858 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90640e7e-f646-4c17-bcbe-d65f04ea70c7 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:21.232299 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbcb607-a779-4f42-82ea-2dcd369c717a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:21.233755 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbcb607-a779-4f42-82ea-2dcd369c717a None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:21.245170 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f431b7f-f0bf-4ece-9b01-76a427dffcfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:21.245977 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f431b7f-f0bf-4ece-9b01-76a427dffcfc None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:21.246256 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f431b7f-f0bf-4ece-9b01-76a427dffcfc None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:21.246525 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f431b7f-f0bf-4ece-9b01-76a427dffcfc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:21.252970 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f431b7f-f0bf-4ece-9b01-76a427dffcfc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:21.254387 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f431b7f-f0bf-4ece-9b01-76a427dffcfc None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:21.269694 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-8f431b7f-f0bf-4ece-9b01-76a427dffcfc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 22:26:21.269694 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:21.269694 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:21.269694 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:21.269694 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:21.269694 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:21.269694 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:21.269694 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:21.269694 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:21.269694 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:21.269694 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:21.269694 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:21.269694 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 22:26:21.269694 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 22:26:21.269694 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 22:26:21.270674 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 22:26:21.270674 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:21.270674 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:21.270674 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:21.270674 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:21.270674 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:21.270674 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:21.270674 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:21.270674 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:21.270674 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:21.270674 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:21.270674 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:21.270674 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:21.270674 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:21.270674 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:21.270674 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:21.270674 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:21.272495 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 22:26:21.272495 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 22:26:21.272495 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 22:26:21.272495 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 22:26:21.272495 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 22:26:21.272495 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 22:26:21.272495 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 22:26:21.272495 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:26:21.272495 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 249/521] 10.222.0.81 () {56 vars in 1111 bytes} [Wed Jul 20 22:26:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:21.272495 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f431b7f-f0bf-4ece-9b01-76a427dffcfc None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:21.332923 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 273/522] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Jul 20 22:26:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 22:26:21.334614 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbcb607-a779-4f42-82ea-2dcd369c717a None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:21.371381 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f431b7f-f0bf-4ece-9b01-76a427dffcfc None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:21.372892 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:21.385529 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:21.404943 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08242bc-b032-4e1b-8bea-da1d52a3f0f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:21.405759 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08242bc-b032-4e1b-8bea-da1d52a3f0f5 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:21.406048 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08242bc-b032-4e1b-8bea-da1d52a3f0f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:21.406327 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08242bc-b032-4e1b-8bea-da1d52a3f0f5 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:21.414913 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08242bc-b032-4e1b-8bea-da1d52a3f0f5 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:21.416727 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08242bc-b032-4e1b-8bea-da1d52a3f0f5 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:21.432775 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-b08242bc-b032-4e1b-8bea-da1d52a3f0f5 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 20 22:26:21.432775 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:21.432775 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:21.432775 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:21.432775 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:21.432775 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:21.432775 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:21.432775 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:21.432775 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:21.432775 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:21.432775 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:21.432775 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:21.432775 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 22:26:21.432775 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 22:26:21.432775 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 22:26:21.432775 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 22:26:21.433816 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:21.433816 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:21.433816 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 22:26:21.433816 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 22:26:21.433816 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 22:26:21.433816 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 22:26:21.433816 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:21.433816 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:21.433816 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:21.433816 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:21.433816 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:21.433816 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:21.433816 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:21.433816 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:21.433816 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:21.433816 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:21.433816 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:21.433816 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:21.439662 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:21.439662 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:21.439662 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 22:26:21.439662 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 22:26:21.439662 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 22:26:21.439662 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 22:26:21.439662 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 22:26:21.439662 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 22:26:21.439662 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 20 22:26:21.439662 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:26:21.440271 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbcb607-a779-4f42-82ea-2dcd369c717a None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:21.440354 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 250/523] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Jul 20 22:26:21 2022] GET /identity/v3/users/cinder => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:21.440354 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08242bc-b032-4e1b-8bea-da1d52a3f0f5 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:21.474031 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff499211-5002-4392-837a-53e88f4abf91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:21.475380 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff499211-5002-4392-837a-53e88f4abf91 None admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:21.475667 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff499211-5002-4392-837a-53e88f4abf91 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:21.475941 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff499211-5002-4392-837a-53e88f4abf91 None admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:21.476635 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff499211-5002-4392-837a-53e88f4abf91 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:21.478086 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff499211-5002-4392-837a-53e88f4abf91 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:21.531789 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08242bc-b032-4e1b-8bea-da1d52a3f0f5 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:21.547296 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 274/524] 10.222.0.81 () {60 vars in 1132 bytes} [Wed Jul 20 22:26:21 2022] POST /identity/v3/users => generated 247 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:26:21.548111 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff499211-5002-4392-837a-53e88f4abf91 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:21.574834 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfd2024-883b-4df5-86b7-7e2666a76c20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:21.575806 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfd2024-883b-4df5-86b7-7e2666a76c20 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:21.576214 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfd2024-883b-4df5-86b7-7e2666a76c20 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:21.576573 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfd2024-883b-4df5-86b7-7e2666a76c20 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:21.585348 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfd2024-883b-4df5-86b7-7e2666a76c20 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:21.586753 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfd2024-883b-4df5-86b7-7e2666a76c20 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:21.603267 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-dbfd2024-883b-4df5-86b7-7e2666a76c20 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 22:26:21.603267 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:21.603267 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:21.603267 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:21.603267 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:21.603267 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:21.603267 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:21.603267 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:21.603267 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:21.603267 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:21.603267 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:21.603267 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:21.603267 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 22:26:21.603267 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 22:26:21.603267 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 22:26:21.604360 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 22:26:21.604360 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:21.604360 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:21.604360 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:21.604360 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:21.604360 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:21.604360 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:21.604360 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:21.604360 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:21.604360 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:21.604360 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:21.604360 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:21.604360 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:21.604360 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:21.604360 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:21.604360 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:21.604360 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:21.605537 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 22:26:21.605537 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 22:26:21.605537 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 22:26:21.605537 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 22:26:21.605537 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 22:26:21.605537 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 22:26:21.605537 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 22:26:21.605537 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:26:21.606632 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 251/525] 10.222.0.81 () {56 vars in 1111 bytes} [Wed Jul 20 22:26:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:21.607573 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfd2024-883b-4df5-86b7-7e2666a76c20 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:21.635890 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:21.679857 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff499211-5002-4392-837a-53e88f4abf91 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:21.689583 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:21.720932 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0793986a-cd8a-4160-99be-a81682d17655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:21.721787 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0793986a-cd8a-4160-99be-a81682d17655 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:21.722149 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0793986a-cd8a-4160-99be-a81682d17655 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:21.722505 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0793986a-cd8a-4160-99be-a81682d17655 None admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:21.723402 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0793986a-cd8a-4160-99be-a81682d17655 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:21.723594 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfd2024-883b-4df5-86b7-7e2666a76c20 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:21.723858 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0793986a-cd8a-4160-99be-a81682d17655 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:21.726159 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0793986a-cd8a-4160-99be-a81682d17655 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:21.745500 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 275/526] 10.222.0.81 () {56 vars in 1148 bytes} [Wed Jul 20 22:26:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:21.746482 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0793986a-cd8a-4160-99be-a81682d17655 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:21.774676 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6c45c9-75d8-47f5-ad79-5c563b2d1a46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:21.775999 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6c45c9-75d8-47f5-ad79-5c563b2d1a46 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:21.776291 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6c45c9-75d8-47f5-ad79-5c563b2d1a46 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:21.776576 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6c45c9-75d8-47f5-ad79-5c563b2d1a46 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:21.788858 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6c45c9-75d8-47f5-ad79-5c563b2d1a46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:21.790316 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6c45c9-75d8-47f5-ad79-5c563b2d1a46 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:21.811433 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-2b6c45c9-75d8-47f5-ad79-5c563b2d1a46 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:26:21.811433 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:21.811433 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:21.811433 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:21.811433 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:21.811433 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:21.811433 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:21.811433 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:21.811433 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:21.811433 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:21.811433 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:21.811433 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:21.811433 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 22:26:21.811433 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 22:26:21.811433 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 22:26:21.812502 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 22:26:21.812502 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:21.812502 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:21.812502 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:21.812502 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:21.812502 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:21.812502 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:21.812502 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:21.812502 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:21.812502 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:21.812502 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:21.812502 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:21.812502 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:21.812502 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:21.812502 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:21.812502 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:21.812502 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:21.813585 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 22:26:21.813585 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 22:26:21.813585 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:26:21.813585 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:26:21.814458 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 252/527] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Jul 20 22:26:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:21.819934 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6c45c9-75d8-47f5-ad79-5c563b2d1a46 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:21.873842 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0793986a-cd8a-4160-99be-a81682d17655 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:21.913760 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:21.917078 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1118dc-79e9-4a46-a023-ebb6401028d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:21.917622 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1118dc-79e9-4a46-a023-ebb6401028d0 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:21.917904 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1118dc-79e9-4a46-a023-ebb6401028d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:21.918344 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1118dc-79e9-4a46-a023-ebb6401028d0 None admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:21.919073 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1118dc-79e9-4a46-a023-ebb6401028d0 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:21.919338 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1118dc-79e9-4a46-a023-ebb6401028d0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:21.920772 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1118dc-79e9-4a46-a023-ebb6401028d0 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:21.944177 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 276/528] 10.222.0.81 () {56 vars in 1137 bytes} [Wed Jul 20 22:26:21 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:21.944382 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6c45c9-75d8-47f5-ad79-5c563b2d1a46 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:21.944479 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1118dc-79e9-4a46-a023-ebb6401028d0 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:21.973096 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:21.979599 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725c6a5b-7ee2-43da-924c-e8c0c6cae819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:21.980368 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725c6a5b-7ee2-43da-924c-e8c0c6cae819 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:21.980714 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725c6a5b-7ee2-43da-924c-e8c0c6cae819 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:21.984405 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725c6a5b-7ee2-43da-924c-e8c0c6cae819 None admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:21.984405 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725c6a5b-7ee2-43da-924c-e8c0c6cae819 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:21.984405 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725c6a5b-7ee2-43da-924c-e8c0c6cae819 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:21.984405 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725c6a5b-7ee2-43da-924c-e8c0c6cae819 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:22.002237 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 253/529] 10.222.0.81 () {56 vars in 1148 bytes} [Wed Jul 20 22:26:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:22.003238 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725c6a5b-7ee2-43da-924c-e8c0c6cae819 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:22.058285 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1118dc-79e9-4a46-a023-ebb6401028d0 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:22.102249 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d13609a-04e0-4294-ace3-e08c3e394e32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:22.102914 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d13609a-04e0-4294-ace3-e08c3e394e32 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:22.103232 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d13609a-04e0-4294-ace3-e08c3e394e32 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:22.103516 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d13609a-04e0-4294-ace3-e08c3e394e32 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:22.104360 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d13609a-04e0-4294-ace3-e08c3e394e32 None admin] RBAC: Adding query filter params (role.id=9b4a83e2eca44a8da9f356bc40f8788a, scope.project.id=385edf2e59e546d090ace36600e29032, user.id=7829324db2574d38b18c749d0559177f) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:22.104679 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d13609a-04e0-4294-ace3-e08c3e394e32 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:22.106121 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d13609a-04e0-4294-ace3-e08c3e394e32 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:22.111875 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 277/530] 10.222.0.81 () {56 vars in 1374 bytes} [Wed Jul 20 22:26:21 2022] GET /identity/v3/role_assignments?user.id=7829324db2574d38b18c749d0559177f&scope.project.id=385edf2e59e546d090ace36600e29032&role.id=9b4a83e2eca44a8da9f356bc40f8788a => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:22.112817 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d13609a-04e0-4294-ace3-e08c3e394e32 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:22.131908 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725c6a5b-7ee2-43da-924c-e8c0c6cae819 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:22.183607 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afd66ad-86ce-4cad-ba4e-e1f940aeadfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:22.184310 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afd66ad-86ce-4cad-ba4e-e1f940aeadfd None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:22.184600 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afd66ad-86ce-4cad-ba4e-e1f940aeadfd None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:22.184879 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afd66ad-86ce-4cad-ba4e-e1f940aeadfd None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:22.185643 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afd66ad-86ce-4cad-ba4e-e1f940aeadfd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:22.185955 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afd66ad-86ce-4cad-ba4e-e1f940aeadfd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:22.188056 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afd66ad-86ce-4cad-ba4e-e1f940aeadfd None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:22.199462 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:22.207188 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 254/531] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Jul 20 22:26:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:22.213660 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afd66ad-86ce-4cad-ba4e-e1f940aeadfd None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:22.230544 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:22.243241 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d13609a-04e0-4294-ace3-e08c3e394e32 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:22.286750 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb24662-3ef5-491f-9e0d-b7d834af3406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:22.287674 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb24662-3ef5-491f-9e0d-b7d834af3406 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:22.287930 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb24662-3ef5-491f-9e0d-b7d834af3406 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:22.288128 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb24662-3ef5-491f-9e0d-b7d834af3406 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:22.333510 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afd66ad-86ce-4cad-ba4e-e1f940aeadfd None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:22.379161 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fab382-d1a4-4d79-ac0c-31a22ed43e9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:22.380202 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fab382-d1a4-4d79-ac0c-31a22ed43e9f None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:22.381259 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fab382-d1a4-4d79-ac0c-31a22ed43e9f None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:22.381732 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fab382-d1a4-4d79-ac0c-31a22ed43e9f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:22.382826 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fab382-d1a4-4d79-ac0c-31a22ed43e9f None admin] RBAC: Adding query filter params (role.id=9b4a83e2eca44a8da9f356bc40f8788a, scope.project.id=385edf2e59e546d090ace36600e29032, user.id=9c44640ea0014426a9523bb1c457778b) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:22.383360 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fab382-d1a4-4d79-ac0c-31a22ed43e9f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:22.385005 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fab382-d1a4-4d79-ac0c-31a22ed43e9f None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:22.389493 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb24662-3ef5-491f-9e0d-b7d834af3406 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:22.391045 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb24662-3ef5-491f-9e0d-b7d834af3406 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:22.392027 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 255/532] 10.222.0.81 () {56 vars in 1374 bytes} [Wed Jul 20 22:26:22 2022] GET /identity/v3/role_assignments?user.id=9c44640ea0014426a9523bb1c457778b&scope.project.id=385edf2e59e546d090ace36600e29032&role.id=9b4a83e2eca44a8da9f356bc40f8788a => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:22.393060 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fab382-d1a4-4d79-ac0c-31a22ed43e9f None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:22.515038 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:22.563752 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:22.575880 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 278/533] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Jul 20 22:26:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 22:26:22.581986 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb24662-3ef5-491f-9e0d-b7d834af3406 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:22.618173 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fab382-d1a4-4d79-ac0c-31a22ed43e9f None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:22.656654 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2528b1f4-3fa7-4d67-9af1-6060ad41514b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:22.657534 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2528b1f4-3fa7-4d67-9af1-6060ad41514b None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:22.657826 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2528b1f4-3fa7-4d67-9af1-6060ad41514b None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:22.658101 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2528b1f4-3fa7-4d67-9af1-6060ad41514b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:22.666989 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2528b1f4-3fa7-4d67-9af1-6060ad41514b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:22.669742 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2528b1f4-3fa7-4d67-9af1-6060ad41514b None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:22.687363 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-2528b1f4-3fa7-4d67-9af1-6060ad41514b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 22:26:22.687363 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:22.687363 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:22.687363 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:22.687363 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:22.687363 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:22.687363 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:22.687363 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:22.687363 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:22.687363 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:22.687363 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:22.687363 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:22.687363 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 22:26:22.687363 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 22:26:22.687363 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 22:26:22.688251 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 22:26:22.688251 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:22.688251 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:22.688251 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:22.688251 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:22.688251 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:22.688251 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:22.688251 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:22.688251 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:22.688251 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:22.688251 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:22.688251 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:22.688251 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:22.688251 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:22.688251 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:22.688251 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:22.688251 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:22.689106 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 22:26:22.689106 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 22:26:22.689106 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 22:26:22.689106 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 22:26:22.689106 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 22:26:22.689106 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 22:26:22.689106 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 22:26:22.689106 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:26:22.689497 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 256/534] 10.222.0.81 () {56 vars in 1111 bytes} [Wed Jul 20 22:26:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:22.694818 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2528b1f4-3fa7-4d67-9af1-6060ad41514b None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:22.765573 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb24662-3ef5-491f-9e0d-b7d834af3406 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:22.811099 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b074a3ac-a049-419e-8650-a59ea0215385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:22.811928 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b074a3ac-a049-419e-8650-a59ea0215385 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:22.812213 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b074a3ac-a049-419e-8650-a59ea0215385 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:22.812492 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b074a3ac-a049-419e-8650-a59ea0215385 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:22.825185 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b074a3ac-a049-419e-8650-a59ea0215385 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:22.826952 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b074a3ac-a049-419e-8650-a59ea0215385 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:22.848462 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-b074a3ac-a049-419e-8650-a59ea0215385 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:26:22.848462 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:22.848462 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:22.848462 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:22.848462 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:22.848462 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:22.848462 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:22.848462 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:22.848462 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:22.848462 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:22.848462 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:22.848462 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:22.848462 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 22:26:22.848462 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 22:26:22.848462 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 22:26:22.849910 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 22:26:22.849910 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:22.849910 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:22.849910 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:22.849910 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:22.849910 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:22.849910 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:22.849910 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:22.849910 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:22.849910 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:22.849910 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:22.849910 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:22.849910 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:22.849910 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:22.849910 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:22.849910 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:22.849910 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:22.850828 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 22:26:22.850828 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 22:26:22.850828 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:26:22.850828 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:26:22.850828 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 279/535] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Jul 20 22:26:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:22.860037 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b074a3ac-a049-419e-8650-a59ea0215385 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:22.900192 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2528b1f4-3fa7-4d67-9af1-6060ad41514b None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:22.921178 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:22.958754 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d5bea5-77f3-4fa6-804e-2518b91815db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:22.960365 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d5bea5-77f3-4fa6-804e-2518b91815db None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:22.961134 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d5bea5-77f3-4fa6-804e-2518b91815db None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:22.961898 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d5bea5-77f3-4fa6-804e-2518b91815db None admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:22.965315 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d5bea5-77f3-4fa6-804e-2518b91815db None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:22.971063 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d5bea5-77f3-4fa6-804e-2518b91815db None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:22.972622 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d5bea5-77f3-4fa6-804e-2518b91815db None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:22.990534 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:23.000259 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 257/536] 10.222.0.81 () {56 vars in 1148 bytes} [Wed Jul 20 22:26:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:23.005765 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d5bea5-77f3-4fa6-804e-2518b91815db None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:23.082789 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b074a3ac-a049-419e-8650-a59ea0215385 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:23.121481 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9917c15b-841c-4c69-9300-4d9cf574387b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:23.122505 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9917c15b-841c-4c69-9300-4d9cf574387b None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:23.123033 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9917c15b-841c-4c69-9300-4d9cf574387b None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:23.123512 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9917c15b-841c-4c69-9300-4d9cf574387b None admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:23.124760 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9917c15b-841c-4c69-9300-4d9cf574387b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:23.125280 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9917c15b-841c-4c69-9300-4d9cf574387b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:23.127119 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9917c15b-841c-4c69-9300-4d9cf574387b None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:23.141590 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 280/537] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Jul 20 22:26:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:23.146643 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9917c15b-841c-4c69-9300-4d9cf574387b None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:23.214874 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d5bea5-77f3-4fa6-804e-2518b91815db None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:23.256376 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673584d9-df67-4668-b1e3-97a9f28fdf88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:23.257390 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673584d9-df67-4668-b1e3-97a9f28fdf88 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:23.257999 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673584d9-df67-4668-b1e3-97a9f28fdf88 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:23.258501 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673584d9-df67-4668-b1e3-97a9f28fdf88 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:23.259681 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673584d9-df67-4668-b1e3-97a9f28fdf88 None admin] RBAC: Adding query filter params (role.id=9b4a83e2eca44a8da9f356bc40f8788a, scope.project.id=385edf2e59e546d090ace36600e29032, user.id=333c2b1b11cc4c478abfd965a72e9d66) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:23.260191 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673584d9-df67-4668-b1e3-97a9f28fdf88 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:23.262623 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673584d9-df67-4668-b1e3-97a9f28fdf88 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:23.270890 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 258/538] 10.222.0.81 () {56 vars in 1374 bytes} [Wed Jul 20 22:26:23 2022] GET /identity/v3/role_assignments?user.id=333c2b1b11cc4c478abfd965a72e9d66&scope.project.id=385edf2e59e546d090ace36600e29032&role.id=9b4a83e2eca44a8da9f356bc40f8788a => generated 259 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:23.313243 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9917c15b-841c-4c69-9300-4d9cf574387b None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:23.315030 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:23.368444 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bc8fd2-3d1b-4490-bdbd-947c24808119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:23.369268 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bc8fd2-3d1b-4490-bdbd-947c24808119 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:23.369552 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bc8fd2-3d1b-4490-bdbd-947c24808119 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:23.369832 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bc8fd2-3d1b-4490-bdbd-947c24808119 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:23.384088 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bc8fd2-3d1b-4490-bdbd-947c24808119 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:23.386294 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bc8fd2-3d1b-4490-bdbd-947c24808119 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:23.406801 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-09bc8fd2-3d1b-4490-bdbd-947c24808119 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 22:26:23.406801 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:23.406801 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:23.406801 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:23.406801 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:23.406801 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:23.406801 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:23.406801 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:23.406801 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:23.406801 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:23.406801 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:23.406801 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:23.406801 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 22:26:23.406801 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 22:26:23.406801 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 22:26:23.407758 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 22:26:23.407758 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:23.407758 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:23.407758 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:23.407758 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:23.407758 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:23.407758 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:23.407758 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:23.407758 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:23.407758 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:23.407758 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:23.407758 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:23.407758 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:23.407758 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:23.407758 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:23.407758 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:23.407758 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:23.408609 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 22:26:23.408609 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 22:26:23.408609 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 22:26:23.408609 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 22:26:23.408609 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 22:26:23.408609 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 22:26:23.408609 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 22:26:23.408609 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:26:23.417757 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 281/539] 10.222.0.81 () {56 vars in 1111 bytes} [Wed Jul 20 22:26:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 22:26:23.419373 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673584d9-df67-4668-b1e3-97a9f28fdf88 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:23.486858 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:23.563292 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673584d9-df67-4668-b1e3-97a9f28fdf88 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:23.614504 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c43a19-c315-4caa-95f5-3003516b271f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:23.616177 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c43a19-c315-4caa-95f5-3003516b271f None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:23.616661 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c43a19-c315-4caa-95f5-3003516b271f None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:23.617133 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c43a19-c315-4caa-95f5-3003516b271f None admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:23.618142 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c43a19-c315-4caa-95f5-3003516b271f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:23.618662 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c43a19-c315-4caa-95f5-3003516b271f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:23.620340 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c43a19-c315-4caa-95f5-3003516b271f None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:23.640375 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 259/540] 10.222.0.81 () {56 vars in 1148 bytes} [Wed Jul 20 22:26:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:23.646408 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bc8fd2-3d1b-4490-bdbd-947c24808119 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:23.771905 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bc8fd2-3d1b-4490-bdbd-947c24808119 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:23.808285 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fb3b1a-0ec4-4d6b-86f8-fcfb0af8dd44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:23.809347 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fb3b1a-0ec4-4d6b-86f8-fcfb0af8dd44 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:23.809842 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fb3b1a-0ec4-4d6b-86f8-fcfb0af8dd44 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:23.810320 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fb3b1a-0ec4-4d6b-86f8-fcfb0af8dd44 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:23.811409 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fb3b1a-0ec4-4d6b-86f8-fcfb0af8dd44 None admin] RBAC: Adding query filter params (role.id=9b4a83e2eca44a8da9f356bc40f8788a, scope.project.id=385edf2e59e546d090ace36600e29032, user.id=ecf4db289fba4e4a806198c62d31f7c4) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:23.811927 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fb3b1a-0ec4-4d6b-86f8-fcfb0af8dd44 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:23.813564 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fb3b1a-0ec4-4d6b-86f8-fcfb0af8dd44 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:23.820206 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 282/541] 10.222.0.81 () {56 vars in 1374 bytes} [Wed Jul 20 22:26:23 2022] GET /identity/v3/role_assignments?user.id=ecf4db289fba4e4a806198c62d31f7c4&scope.project.id=385edf2e59e546d090ace36600e29032&role.id=9b4a83e2eca44a8da9f356bc40f8788a => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:24.040920 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7da11f-df39-4d9a-9939-2ce8d7db0500 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:24.041947 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7da11f-df39-4d9a-9939-2ce8d7db0500 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:24.042458 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7da11f-df39-4d9a-9939-2ce8d7db0500 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:24.044491 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 260/542] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:26:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:26:24.049781 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f493fa-4f83-4ebb-9afc-c9af22ed0c6d None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:24.050730 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f493fa-4f83-4ebb-9afc-c9af22ed0c6d None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:24.051262 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f493fa-4f83-4ebb-9afc-c9af22ed0c6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:24.074629 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-42f493fa-4f83-4ebb-9afc-c9af22ed0c6d None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:26:24.076199 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-42f493fa-4f83-4ebb-9afc-c9af22ed0c6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:26:24.099117 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:24.156783 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 283/543] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:26:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 22:26:24.198415 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7da11f-df39-4d9a-9939-2ce8d7db0500 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:24.336420 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7da11f-df39-4d9a-9939-2ce8d7db0500 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:24.338176 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:24.387723 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9e8f48-7d7d-4021-8146-95b327dc44d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:24.388562 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9e8f48-7d7d-4021-8146-95b327dc44d6 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:24.388929 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9e8f48-7d7d-4021-8146-95b327dc44d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:24.389213 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9e8f48-7d7d-4021-8146-95b327dc44d6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:24.404096 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9e8f48-7d7d-4021-8146-95b327dc44d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 22:26:24.413030 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-bb9e8f48-7d7d-4021-8146-95b327dc44d6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 22:26:24.413030 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:24.413030 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:24.413030 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:24.413030 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:24.413030 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:24.413030 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:24.413030 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:24.413030 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:24.413030 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:24.413030 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:24.413030 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:24.413030 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 22:26:24.413030 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 22:26:24.413030 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 22:26:24.413030 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 22:26:24.413915 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 22:26:24.413915 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 22:26:24.413915 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 22:26:24.413915 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 22:26:24.413915 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:24.413915 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:24.413915 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:24.413915 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:24.413915 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:24.413915 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:24.413915 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:24.413915 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:24.413915 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:24.413915 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:24.413915 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:24.413915 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:24.414702 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:24.414702 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:24.414702 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:24.414702 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:24.414702 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 22:26:24.414702 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 22:26:24.414702 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 22:26:24.414702 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 22:26:24.414702 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 22:26:24.414702 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 22:26:24.414702 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 22:26:24.414702 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:26:24.414702 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 261/544] 10.222.0.81 () {56 vars in 1102 bytes} [Wed Jul 20 22:26:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:24.422075 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f493fa-4f83-4ebb-9afc-c9af22ed0c6d None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:24.538934 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e2b400-ac61-4a05-a59e-cf0d7cfa0b1e None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:24.539423 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e2b400-ac61-4a05-a59e-cf0d7cfa0b1e None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:24.540369 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e2b400-ac61-4a05-a59e-cf0d7cfa0b1e None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:24.542300 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 262/545] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:26:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:26:24.553967 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2f9e1d-3956-4c5e-ac17-77c83743776e None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:24.554284 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2f9e1d-3956-4c5e-ac17-77c83743776e None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:24.554567 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2f9e1d-3956-4c5e-ac17-77c83743776e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:24.585199 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-ea2f9e1d-3956-4c5e-ac17-77c83743776e None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:26:24.586480 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-ea2f9e1d-3956-4c5e-ac17-77c83743776e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:26:24.605901 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f493fa-4f83-4ebb-9afc-c9af22ed0c6d None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:24.664052 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faea18bf-23a5-4297-9bf4-cc820598214b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:24.664739 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faea18bf-23a5-4297-9bf4-cc820598214b None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:24.665029 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faea18bf-23a5-4297-9bf4-cc820598214b None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:24.665565 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faea18bf-23a5-4297-9bf4-cc820598214b None admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:24.666324 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faea18bf-23a5-4297-9bf4-cc820598214b None admin] RBAC: Adding query filter params (name=service) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:24.666636 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faea18bf-23a5-4297-9bf4-cc820598214b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:24.668048 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faea18bf-23a5-4297-9bf4-cc820598214b None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:24.676402 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 284/546] 10.222.0.81 () {56 vars in 1103 bytes} [Wed Jul 20 22:26:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:24.677874 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e631f337-98d6-4c79-bc82-e69fda52f0cb None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:24.678174 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e631f337-98d6-4c79-bc82-e69fda52f0cb None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:24.678350 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 263/547] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:26:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jul 20 22:26:24.678517 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e631f337-98d6-4c79-bc82-e69fda52f0cb None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:24.683974 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 285/548] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:26:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:26:24.685799 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5816a3d3-9948-4844-b5a4-0f4b4ddec659 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:24.686098 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5816a3d3-9948-4844-b5a4-0f4b4ddec659 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:24.686384 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5816a3d3-9948-4844-b5a4-0f4b4ddec659 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:24.688584 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e631f337-98d6-4c79-bc82-e69fda52f0cb None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:24.712232 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-5816a3d3-9948-4844-b5a4-0f4b4ddec659 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:26:24.713488 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-5816a3d3-9948-4844-b5a4-0f4b4ddec659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:26:24.714639 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:24.791247 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 264/549] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:26:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 22:26:24.792268 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5816a3d3-9948-4844-b5a4-0f4b4ddec659 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:24.801500 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e631f337-98d6-4c79-bc82-e69fda52f0cb None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:24.824076 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:24.869831 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195d816f-27e7-44c5-b15f-22f5ca902239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:24.870687 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195d816f-27e7-44c5-b15f-22f5ca902239 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:24.870981 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195d816f-27e7-44c5-b15f-22f5ca902239 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:24.871304 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195d816f-27e7-44c5-b15f-22f5ca902239 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:24.883764 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195d816f-27e7-44c5-b15f-22f5ca902239 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:24.885289 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195d816f-27e7-44c5-b15f-22f5ca902239 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:24.905174 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-195d816f-27e7-44c5-b15f-22f5ca902239 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:26:24.905174 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:24.905174 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:24.905174 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:24.905174 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:24.905174 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:24.905174 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:24.905174 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:24.905174 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:24.905174 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:24.905174 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:24.905174 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:24.905174 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 22:26:24.905174 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 22:26:24.905174 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 22:26:24.906032 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 22:26:24.906032 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:24.906032 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:24.906032 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:24.906032 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:24.906032 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:24.906032 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:24.906032 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:24.906032 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:24.906032 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:24.906032 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:24.906032 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:24.906032 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:24.906032 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:24.906032 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:24.906032 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:24.906032 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:24.907425 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 22:26:24.907425 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 22:26:24.907425 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:26:24.907425 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:26:24.907425 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 286/550] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Jul 20 22:26:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:24.907866 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195d816f-27e7-44c5-b15f-22f5ca902239 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:24.921965 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5816a3d3-9948-4844-b5a4-0f4b4ddec659 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:24.970208 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2a4bcd-1cb6-41fc-a045-b96e01860d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:24.971570 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2a4bcd-1cb6-41fc-a045-b96e01860d31 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:24.971864 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2a4bcd-1cb6-41fc-a045-b96e01860d31 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:24.972290 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2a4bcd-1cb6-41fc-a045-b96e01860d31 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:24.986846 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2a4bcd-1cb6-41fc-a045-b96e01860d31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 22:26:24.992157 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:24.997700 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-db2a4bcd-1cb6-41fc-a045-b96e01860d31 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 22:26:24.997700 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:24.997700 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:24.997700 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:24.997700 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:24.997700 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:24.997700 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:24.997700 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:24.997700 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:24.997700 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:24.997700 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:24.997700 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:24.997700 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 22:26:24.997700 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 22:26:24.997700 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 22:26:24.997700 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 22:26:24.999797 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 22:26:24.999797 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 22:26:24.999797 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 22:26:24.999797 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 22:26:24.999797 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:24.999797 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:24.999797 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:24.999797 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:24.999797 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:24.999797 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:24.999797 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:24.999797 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:24.999797 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:24.999797 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:24.999797 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:24.999797 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:25.001196 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:25.001196 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:25.001196 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:25.001196 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:25.001196 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 22:26:25.001196 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 22:26:25.001196 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 22:26:25.001196 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 22:26:25.001196 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 22:26:25.001196 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 22:26:25.001196 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 22:26:25.001196 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:26:25.004401 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 265/551] 10.222.0.81 () {56 vars in 1102 bytes} [Wed Jul 20 22:26:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:25.004401 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2a4bcd-1cb6-41fc-a045-b96e01860d31 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:25.032727 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195d816f-27e7-44c5-b15f-22f5ca902239 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:25.077675 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcac2e59-c92e-4ba6-a40e-c7f39bcc4482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:25.078513 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcac2e59-c92e-4ba6-a40e-c7f39bcc4482 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:25.078804 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcac2e59-c92e-4ba6-a40e-c7f39bcc4482 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:25.079126 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcac2e59-c92e-4ba6-a40e-c7f39bcc4482 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:25.090221 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcac2e59-c92e-4ba6-a40e-c7f39bcc4482 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 22:26:25.096690 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:25.100028 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-bcac2e59-c92e-4ba6-a40e-c7f39bcc4482 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 22:26:25.100028 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:25.100028 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:25.100028 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:25.100028 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:25.100028 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:25.100028 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:25.100028 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:25.100028 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:25.100028 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:25.100028 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:25.100028 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:25.100028 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 22:26:25.100028 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 22:26:25.100028 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 22:26:25.100028 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 22:26:25.102508 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 22:26:25.102508 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 22:26:25.102508 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 22:26:25.102508 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 22:26:25.102508 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:25.102508 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:25.102508 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:25.102508 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:25.102508 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:25.102508 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:25.102508 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:25.102508 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:25.102508 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:25.102508 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:25.102508 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:25.102508 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:25.104512 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:25.104512 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:25.104512 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:25.104512 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:25.104512 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 22:26:25.104512 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 22:26:25.104512 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 22:26:25.104512 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 22:26:25.104512 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 22:26:25.104512 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 22:26:25.104512 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 22:26:25.104512 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:26:25.104512 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 287/552] 10.222.0.81 () {56 vars in 1102 bytes} [Wed Jul 20 22:26:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:25.104512 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcac2e59-c92e-4ba6-a40e-c7f39bcc4482 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:25.121830 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2a4bcd-1cb6-41fc-a045-b96e01860d31 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:25.156651 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcc190c-a7be-4a00-b2cb-764a2fc96aaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:25.157330 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcc190c-a7be-4a00-b2cb-764a2fc96aaa None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:25.157620 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcc190c-a7be-4a00-b2cb-764a2fc96aaa None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:25.157900 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcc190c-a7be-4a00-b2cb-764a2fc96aaa None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:25.158619 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcc190c-a7be-4a00-b2cb-764a2fc96aaa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:25.158930 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcc190c-a7be-4a00-b2cb-764a2fc96aaa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:25.160442 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcc190c-a7be-4a00-b2cb-764a2fc96aaa None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:25.173910 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 266/553] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Jul 20 22:26:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:25.174857 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcc190c-a7be-4a00-b2cb-764a2fc96aaa None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:25.211446 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcac2e59-c92e-4ba6-a40e-c7f39bcc4482 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:25.243660 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:25.250903 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20180fd-35ba-4777-b8ad-0e21e077544d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:25.251609 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20180fd-35ba-4777-b8ad-0e21e077544d None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:25.252162 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20180fd-35ba-4777-b8ad-0e21e077544d None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:25.252451 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20180fd-35ba-4777-b8ad-0e21e077544d None admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:25.255793 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20180fd-35ba-4777-b8ad-0e21e077544d None admin] RBAC: Adding query filter params (name=service) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:25.255793 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20180fd-35ba-4777-b8ad-0e21e077544d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:25.255793 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20180fd-35ba-4777-b8ad-0e21e077544d None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:25.269470 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 288/554] 10.222.0.81 () {56 vars in 1103 bytes} [Wed Jul 20 22:26:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:25.270371 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20180fd-35ba-4777-b8ad-0e21e077544d None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:25.290358 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcc190c-a7be-4a00-b2cb-764a2fc96aaa None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:25.332051 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6836f312-db72-49aa-9f3f-e42dd134688c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:25.333090 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6836f312-db72-49aa-9f3f-e42dd134688c None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:25.334062 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6836f312-db72-49aa-9f3f-e42dd134688c None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:25.334483 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6836f312-db72-49aa-9f3f-e42dd134688c None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:25.335431 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6836f312-db72-49aa-9f3f-e42dd134688c None admin] RBAC: Adding query filter params (name=service) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:25.335903 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6836f312-db72-49aa-9f3f-e42dd134688c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:25.337422 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6836f312-db72-49aa-9f3f-e42dd134688c None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:25.346119 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 267/555] 10.222.0.81 () {56 vars in 1103 bytes} [Wed Jul 20 22:26:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:25.347274 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6836f312-db72-49aa-9f3f-e42dd134688c None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:25.352353 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:25.384269 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20180fd-35ba-4777-b8ad-0e21e077544d None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:25.423341 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f0ce49-4d3e-4a70-bf40-6e61e8a7be38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:25.424212 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f0ce49-4d3e-4a70-bf40-6e61e8a7be38 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:25.424507 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f0ce49-4d3e-4a70-bf40-6e61e8a7be38 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:25.424790 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f0ce49-4d3e-4a70-bf40-6e61e8a7be38 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:25.432917 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f0ce49-4d3e-4a70-bf40-6e61e8a7be38 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:25.434498 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f0ce49-4d3e-4a70-bf40-6e61e8a7be38 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:25.451730 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-88f0ce49-4d3e-4a70-bf40-6e61e8a7be38 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 20 22:26:25.451730 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:25.451730 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:25.451730 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:25.451730 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:25.451730 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:25.451730 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:25.451730 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:25.451730 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:25.451730 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:25.451730 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:25.451730 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:25.451730 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 22:26:25.451730 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 22:26:25.451730 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 22:26:25.451730 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 22:26:25.452691 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:25.452691 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:25.452691 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 22:26:25.452691 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 22:26:25.452691 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 22:26:25.452691 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 22:26:25.452691 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:25.452691 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:25.452691 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:25.452691 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:25.452691 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:25.452691 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:25.452691 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:25.452691 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:25.452691 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:25.452691 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:25.452691 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:25.452691 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:25.453654 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:25.453654 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:25.453654 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 22:26:25.453654 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 22:26:25.453654 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 22:26:25.453654 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 22:26:25.453654 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 22:26:25.453654 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 22:26:25.453654 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 20 22:26:25.453654 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:26:25.453654 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 289/556] 10.222.0.81 () {56 vars in 1096 bytes} [Wed Jul 20 22:26:25 2022] GET /identity/v3/users/swift => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:25.454229 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f0ce49-4d3e-4a70-bf40-6e61e8a7be38 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:25.455509 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6836f312-db72-49aa-9f3f-e42dd134688c None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:25.491343 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642cf036-fe63-402c-9ea1-4287bf9b467b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:25.492519 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642cf036-fe63-402c-9ea1-4287bf9b467b None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:25.493006 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642cf036-fe63-402c-9ea1-4287bf9b467b None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:25.493503 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642cf036-fe63-402c-9ea1-4287bf9b467b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:25.504472 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642cf036-fe63-402c-9ea1-4287bf9b467b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:25.506789 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642cf036-fe63-402c-9ea1-4287bf9b467b None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:25.524559 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-642cf036-fe63-402c-9ea1-4287bf9b467b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:26:25.524559 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:25.524559 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:25.524559 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:25.524559 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:25.524559 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:25.524559 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:25.524559 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:25.524559 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:25.524559 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:25.524559 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:25.524559 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:25.524559 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 22:26:25.524559 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 22:26:25.524559 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 22:26:25.525444 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 22:26:25.525444 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:25.525444 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:25.525444 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:25.525444 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:25.525444 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:25.525444 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:25.525444 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:25.525444 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:25.525444 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:25.525444 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:25.525444 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:25.525444 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:25.525444 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:25.525444 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:25.525444 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:25.525444 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:25.526797 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 22:26:25.526797 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 22:26:25.526797 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:26:25.526797 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:26:25.527447 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:25.529204 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 268/557] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Jul 20 22:26:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:25.530202 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642cf036-fe63-402c-9ea1-4287bf9b467b None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:25.570396 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f0ce49-4d3e-4a70-bf40-6e61e8a7be38 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:25.590505 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:25.607258 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc23cb9-2c7a-4c15-b12c-56d1f9528d6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:25.608128 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc23cb9-2c7a-4c15-b12c-56d1f9528d6f None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:25.608419 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc23cb9-2c7a-4c15-b12c-56d1f9528d6f None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:25.608703 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc23cb9-2c7a-4c15-b12c-56d1f9528d6f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:25.621510 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc23cb9-2c7a-4c15-b12c-56d1f9528d6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:25.622970 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc23cb9-2c7a-4c15-b12c-56d1f9528d6f None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:25.639779 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642cf036-fe63-402c-9ea1-4287bf9b467b None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:25.643813 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-cbc23cb9-2c7a-4c15-b12c-56d1f9528d6f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:26:25.643813 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:25.643813 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:25.643813 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:25.643813 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:25.643813 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:25.643813 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:25.643813 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:25.643813 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:25.643813 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:25.643813 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:25.643813 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:25.643813 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 22:26:25.643813 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 22:26:25.643813 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 22:26:25.644715 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 22:26:25.644715 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:25.644715 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:25.644715 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:25.644715 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:25.644715 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:25.644715 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:25.644715 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:25.644715 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:25.644715 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:25.644715 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:25.644715 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:25.644715 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:25.644715 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:25.644715 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:25.644715 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:25.644715 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:25.645669 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 22:26:25.645669 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 22:26:25.645669 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:26:25.645669 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:26:25.645669 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 290/558] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Jul 20 22:26:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:25.646468 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc23cb9-2c7a-4c15-b12c-56d1f9528d6f None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:25.688087 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8b9d9a-a112-4766-8eab-cbefba23ce25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:25.689114 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8b9d9a-a112-4766-8eab-cbefba23ce25 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:25.689502 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8b9d9a-a112-4766-8eab-cbefba23ce25 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:25.689764 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8b9d9a-a112-4766-8eab-cbefba23ce25 None admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:25.690623 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8b9d9a-a112-4766-8eab-cbefba23ce25 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:25.690957 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8b9d9a-a112-4766-8eab-cbefba23ce25 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:25.692495 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8b9d9a-a112-4766-8eab-cbefba23ce25 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:25.711733 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 269/559] 10.222.0.81 () {56 vars in 1135 bytes} [Wed Jul 20 22:26:25 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:25.712859 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8b9d9a-a112-4766-8eab-cbefba23ce25 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:25.772070 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc23cb9-2c7a-4c15-b12c-56d1f9528d6f None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:25.775613 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:25.819929 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a65abc4-c0ea-4414-9039-95c48cb1478f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:25.820601 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a65abc4-c0ea-4414-9039-95c48cb1478f None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:25.820892 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a65abc4-c0ea-4414-9039-95c48cb1478f None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:25.821175 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a65abc4-c0ea-4414-9039-95c48cb1478f None admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:25.821978 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a65abc4-c0ea-4414-9039-95c48cb1478f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:25.822292 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a65abc4-c0ea-4414-9039-95c48cb1478f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:25.823788 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a65abc4-c0ea-4414-9039-95c48cb1478f None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:25.826727 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8b9d9a-a112-4766-8eab-cbefba23ce25 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:25.840291 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 291/560] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Jul 20 22:26:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:25.841208 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a65abc4-c0ea-4414-9039-95c48cb1478f None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:25.844371 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:25.871132 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24f556d-d5ff-417f-90f4-8de7e52f7b11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:25.872765 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24f556d-d5ff-417f-90f4-8de7e52f7b11 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:25.873267 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24f556d-d5ff-417f-90f4-8de7e52f7b11 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:25.873878 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24f556d-d5ff-417f-90f4-8de7e52f7b11 None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:25.875407 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24f556d-d5ff-417f-90f4-8de7e52f7b11 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:25.875945 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24f556d-d5ff-417f-90f4-8de7e52f7b11 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:25.877712 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24f556d-d5ff-417f-90f4-8de7e52f7b11 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:25.889521 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 270/561] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Jul 20 22:26:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:25.890598 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24f556d-d5ff-417f-90f4-8de7e52f7b11 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:25.950577 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a65abc4-c0ea-4414-9039-95c48cb1478f None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:25.987734 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ec7547-0ddc-4f1b-9f5c-924cc5339e32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:25.988385 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ec7547-0ddc-4f1b-9f5c-924cc5339e32 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:25.988688 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ec7547-0ddc-4f1b-9f5c-924cc5339e32 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:25.988953 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ec7547-0ddc-4f1b-9f5c-924cc5339e32 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:25.996148 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24f556d-d5ff-417f-90f4-8de7e52f7b11 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:26.023648 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:26.034265 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ba9fd1-362b-46bb-986c-f3544d226f01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:26.035123 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ba9fd1-362b-46bb-986c-f3544d226f01 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:26.035413 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ba9fd1-362b-46bb-986c-f3544d226f01 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:26.035932 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ba9fd1-362b-46bb-986c-f3544d226f01 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:26.044642 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ba9fd1-362b-46bb-986c-f3544d226f01 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:26.051317 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ba9fd1-362b-46bb-986c-f3544d226f01 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:26.062474 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-75ba9fd1-362b-46bb-986c-f3544d226f01 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 20 22:26:26.062474 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:26.062474 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:26.062474 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:26.062474 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:26.062474 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:26.062474 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:26.062474 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:26.062474 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:26.062474 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:26.062474 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:26.062474 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:26.062474 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 22:26:26.062474 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 22:26:26.062474 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 22:26:26.062474 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 22:26:26.063697 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:26.063697 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:26.063697 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 22:26:26.063697 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 22:26:26.063697 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 22:26:26.063697 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 22:26:26.063697 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:26.063697 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:26.063697 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:26.063697 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:26.063697 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:26.063697 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:26.063697 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:26.063697 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:26.063697 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:26.063697 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:26.063697 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:26.063697 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:26.064649 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:26.064649 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:26.064649 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 22:26:26.064649 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 22:26:26.064649 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 22:26:26.064649 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 22:26:26.064649 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 22:26:26.064649 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 22:26:26.064649 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 20 22:26:26.064649 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:26:26.064649 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 271/562] 10.222.0.81 () {56 vars in 1093 bytes} [Wed Jul 20 22:26:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:26.065298 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3356d9a8-5c09-4a97-a85a-80196c281554 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:26.065298 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3356d9a8-5c09-4a97-a85a-80196c281554 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:26.065433 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3356d9a8-5c09-4a97-a85a-80196c281554 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:26.071366 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 272/563] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:26:26 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:26:26.071366 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3356d9a8-5c09-4a97-a85a-80196c281554 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:26.073060 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ec7547-0ddc-4f1b-9f5c-924cc5339e32 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:26.074669 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ec7547-0ddc-4f1b-9f5c-924cc5339e32 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:26.106748 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:26.177689 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 292/564] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Jul 20 22:26:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 22:26:26.178205 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3356d9a8-5c09-4a97-a85a-80196c281554 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:26.179208 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ec7547-0ddc-4f1b-9f5c-924cc5339e32 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:26.217965 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5e570e-6438-4455-9e4b-130060ab058b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:26.219065 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5e570e-6438-4455-9e4b-130060ab058b None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:26.219357 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5e570e-6438-4455-9e4b-130060ab058b None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:26.219638 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5e570e-6438-4455-9e4b-130060ab058b None admin] PATH_INFO: `/v3/users/glance` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:26.228010 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5e570e-6438-4455-9e4b-130060ab058b None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:26.229467 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5e570e-6438-4455-9e4b-130060ab058b None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:26.244206 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-7c5e570e-6438-4455-9e4b-130060ab058b None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 20 22:26:26.244206 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:26.244206 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:26.244206 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:26.244206 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:26.244206 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:26.244206 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:26.244206 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:26.244206 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:26.244206 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:26.244206 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:26.244206 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:26.244206 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 22:26:26.244206 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 22:26:26.244206 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 22:26:26.244206 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 22:26:26.245151 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:26.245151 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:26.245151 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 22:26:26.245151 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 22:26:26.245151 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 22:26:26.245151 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 22:26:26.245151 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:26.245151 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:26.245151 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:26.245151 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:26.245151 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:26.245151 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:26.245151 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:26.245151 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:26.245151 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:26.245151 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:26.245151 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:26.245151 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:26.246072 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:26.246072 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:26.246072 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 22:26:26.246072 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 22:26:26.246072 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 22:26:26.246072 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 22:26:26.246072 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 22:26:26.246072 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 22:26:26.246072 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 20 22:26:26.246072 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:26:26.246072 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 273/565] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Jul 20 22:26:26 2022] GET /identity/v3/users/glance => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:26.246660 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5e245c-57fd-4222-9094-565ef3e05d19 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:26.247634 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5e245c-57fd-4222-9094-565ef3e05d19 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:26.247634 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5e245c-57fd-4222-9094-565ef3e05d19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:26.256850 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:26.263731 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-4f5e245c-57fd-4222-9094-565ef3e05d19 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:26:26.265245 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-4f5e245c-57fd-4222-9094-565ef3e05d19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:26:26.296201 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ec7547-0ddc-4f1b-9f5c-924cc5339e32 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:26.329684 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2713f72f-26dc-4fab-aa3e-e72c109ce122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:26.330355 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2713f72f-26dc-4fab-aa3e-e72c109ce122 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:26.330636 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2713f72f-26dc-4fab-aa3e-e72c109ce122 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:26.330912 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2713f72f-26dc-4fab-aa3e-e72c109ce122 None admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:26.331770 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2713f72f-26dc-4fab-aa3e-e72c109ce122 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:26.332078 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2713f72f-26dc-4fab-aa3e-e72c109ce122 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:26.332325 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 274/566] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:26:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 22:26:26.333682 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5e245c-57fd-4222-9094-565ef3e05d19 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:26.334313 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2713f72f-26dc-4fab-aa3e-e72c109ce122 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:26.353541 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 293/567] 10.222.0.81 () {56 vars in 1133 bytes} [Wed Jul 20 22:26:26 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:26.354421 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2713f72f-26dc-4fab-aa3e-e72c109ce122 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:26.360359 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:26.433318 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5e245c-57fd-4222-9094-565ef3e05d19 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:26.454120 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2713f72f-26dc-4fab-aa3e-e72c109ce122 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:26.466824 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174efac4-9400-4b4e-a1a2-dcd272877bfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:26.467675 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174efac4-9400-4b4e-a1a2-dcd272877bfc None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:26.467964 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174efac4-9400-4b4e-a1a2-dcd272877bfc None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:26.468248 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174efac4-9400-4b4e-a1a2-dcd272877bfc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:26.476951 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:26.478623 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174efac4-9400-4b4e-a1a2-dcd272877bfc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:26.480116 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174efac4-9400-4b4e-a1a2-dcd272877bfc None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:26.487810 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551243db-2f7e-4779-ab67-a957851d35b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:26.491130 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551243db-2f7e-4779-ab67-a957851d35b9 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:26.491130 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551243db-2f7e-4779-ab67-a957851d35b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:26.491130 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551243db-2f7e-4779-ab67-a957851d35b9 None admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:26.491130 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551243db-2f7e-4779-ab67-a957851d35b9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:26.491130 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551243db-2f7e-4779-ab67-a957851d35b9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:26.491799 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551243db-2f7e-4779-ab67-a957851d35b9 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:26.497943 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-174efac4-9400-4b4e-a1a2-dcd272877bfc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:26:26.497943 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:26.497943 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:26.497943 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:26.497943 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:26.497943 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:26.497943 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:26.497943 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:26.497943 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:26.497943 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:26.497943 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:26.497943 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:26.497943 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 22:26:26.497943 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 22:26:26.497943 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 22:26:26.498846 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 22:26:26.498846 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:26.498846 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:26.498846 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:26.498846 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:26.498846 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:26.498846 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:26.498846 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:26.498846 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:26.498846 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:26.498846 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:26.498846 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:26.498846 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:26.498846 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:26.498846 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:26.498846 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:26.498846 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:26.499760 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 22:26:26.499760 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 22:26:26.499760 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:26:26.499760 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:26:26.499999 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 275/568] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Jul 20 22:26:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:26.501170 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baceae8c-a614-4315-8713-e1a0227a3ad8 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:26.501456 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baceae8c-a614-4315-8713-e1a0227a3ad8 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:26.501732 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baceae8c-a614-4315-8713-e1a0227a3ad8 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:26.503147 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 276/569] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:26:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:26:26.504054 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baceae8c-a614-4315-8713-e1a0227a3ad8 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:26.515477 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 294/570] 10.222.0.81 () {56 vars in 1137 bytes} [Wed Jul 20 22:26:26 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:26.516149 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551243db-2f7e-4779-ab67-a957851d35b9 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:26.584423 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:26.607302 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baceae8c-a614-4315-8713-e1a0227a3ad8 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:26.616921 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551243db-2f7e-4779-ab67-a957851d35b9 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:26.642422 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53195cd1-11e1-465a-a4ba-f9e48fd88999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:26.643255 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53195cd1-11e1-465a-a4ba-f9e48fd88999 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:26.643547 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53195cd1-11e1-465a-a4ba-f9e48fd88999 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:26.643829 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53195cd1-11e1-465a-a4ba-f9e48fd88999 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:26.652292 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b1feba-59c9-47c5-a3f1-e85927c8d72e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:26.653092 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b1feba-59c9-47c5-a3f1-e85927c8d72e None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:26.653377 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b1feba-59c9-47c5-a3f1-e85927c8d72e None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:26.653660 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b1feba-59c9-47c5-a3f1-e85927c8d72e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:26.654257 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53195cd1-11e1-465a-a4ba-f9e48fd88999 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:26.655734 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53195cd1-11e1-465a-a4ba-f9e48fd88999 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:26.665279 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b1feba-59c9-47c5-a3f1-e85927c8d72e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 22:26:26.673391 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-79b1feba-59c9-47c5-a3f1-e85927c8d72e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 22:26:26.673391 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:26.673391 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:26.673391 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:26.673391 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:26.673391 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:26.673391 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:26.673391 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:26.673391 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:26.673391 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:26.673391 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:26.673391 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:26.673391 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 22:26:26.673391 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 22:26:26.673391 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 22:26:26.673391 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 22:26:26.674444 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 22:26:26.674444 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 22:26:26.674444 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 22:26:26.674444 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 22:26:26.674444 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:26.674444 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:26.674444 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:26.674444 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:26.674444 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:26.674444 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:26.674444 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:26.674444 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:26.674444 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:26.674444 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:26.674444 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:26.674444 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:26.675567 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:26.675567 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:26.675567 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:26.675567 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:26.675567 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 22:26:26.675567 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 22:26:26.675567 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 22:26:26.675567 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 22:26:26.675567 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 22:26:26.675567 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 22:26:26.675567 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 22:26:26.675567 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:26:26.676213 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-53195cd1-11e1-465a-a4ba-f9e48fd88999 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:26:26.676213 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:26.676213 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:26.676213 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:26.676213 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:26.676213 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:26.676213 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:26.676213 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:26.676213 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:26.676213 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:26.676213 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:26.676213 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:26.676213 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 22:26:26.676213 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 22:26:26.676213 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 22:26:26.677550 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 22:26:26.677550 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:26.677550 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:26.677550 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:26.677550 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:26.677550 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:26.677550 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:26.677550 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:26.677550 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:26.677550 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:26.677550 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:26.677550 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:26.677550 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:26.677550 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:26.677550 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:26.677550 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:26.677550 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:26.678423 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 22:26:26.678423 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 22:26:26.678423 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:26:26.678423 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:26:26.678635 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 295/571] 10.222.0.81 () {56 vars in 1102 bytes} [Wed Jul 20 22:26:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:26.678711 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 277/572] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Jul 20 22:26:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:26.678785 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b1feba-59c9-47c5-a3f1-e85927c8d72e None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:26.678854 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4d70b6-10cb-439c-a7b1-f4b73133abb5 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:26.678854 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4d70b6-10cb-439c-a7b1-f4b73133abb5 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:26.678854 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4d70b6-10cb-439c-a7b1-f4b73133abb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:26.694136 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-cb4d70b6-10cb-439c-a7b1-f4b73133abb5 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:26:26.695418 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-cb4d70b6-10cb-439c-a7b1-f4b73133abb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:26:26.721496 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:26.762161 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 278/573] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:26:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 22:26:26.762996 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4d70b6-10cb-439c-a7b1-f4b73133abb5 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:26.792686 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b1feba-59c9-47c5-a3f1-e85927c8d72e None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:26.821457 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:26.827369 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509306bb-c18a-4660-a35e-cedcc4beb452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:26.828076 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509306bb-c18a-4660-a35e-cedcc4beb452 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:26.832762 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509306bb-c18a-4660-a35e-cedcc4beb452 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:26.832762 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509306bb-c18a-4660-a35e-cedcc4beb452 None admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:26.832762 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509306bb-c18a-4660-a35e-cedcc4beb452 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:26.832762 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509306bb-c18a-4660-a35e-cedcc4beb452 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:26.832762 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509306bb-c18a-4660-a35e-cedcc4beb452 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:26.843189 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 296/574] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Jul 20 22:26:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:26.843985 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509306bb-c18a-4660-a35e-cedcc4beb452 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:26.891772 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4d70b6-10cb-439c-a7b1-f4b73133abb5 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:26.925778 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6336e6-b512-4f42-aa07-432d40bb1e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:26.926591 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6336e6-b512-4f42-aa07-432d40bb1e7b None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:26.926932 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6336e6-b512-4f42-aa07-432d40bb1e7b None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:26.927247 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6336e6-b512-4f42-aa07-432d40bb1e7b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:26.937099 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6336e6-b512-4f42-aa07-432d40bb1e7b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:26.938537 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6336e6-b512-4f42-aa07-432d40bb1e7b None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:26.954298 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:26.957021 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509306bb-c18a-4660-a35e-cedcc4beb452 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:26.958465 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-cf6336e6-b512-4f42-aa07-432d40bb1e7b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:26:26.958465 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:26.958465 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:26.958465 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:26.958465 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:26.958465 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:26.958465 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:26.958465 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:26.958465 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:26.958465 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:26.958465 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:26.958465 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:26.958465 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 22:26:26.958465 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 22:26:26.958465 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 22:26:26.959531 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 22:26:26.959531 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:26.959531 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:26.959531 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:26.959531 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:26.959531 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:26.959531 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:26.959531 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:26.959531 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:26.959531 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:26.959531 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:26.959531 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:26.959531 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:26.959531 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:26.959531 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:26.959531 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:26.959531 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:26.960434 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 22:26:26.960434 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 22:26:26.960434 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:26:26.960434 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:26:26.960434 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 279/575] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Jul 20 22:26:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:26.960434 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6336e6-b512-4f42-aa07-432d40bb1e7b None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:26.995597 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7485f66-907b-452f-a8f1-da5d5ada3c64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:26.996253 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7485f66-907b-452f-a8f1-da5d5ada3c64 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:26.996540 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7485f66-907b-452f-a8f1-da5d5ada3c64 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:26.996826 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7485f66-907b-452f-a8f1-da5d5ada3c64 None admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:26.997550 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7485f66-907b-452f-a8f1-da5d5ada3c64 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:26.997864 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7485f66-907b-452f-a8f1-da5d5ada3c64 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:26.999260 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7485f66-907b-452f-a8f1-da5d5ada3c64 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:27.011158 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 297/576] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Jul 20 22:26:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:27.011955 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7485f66-907b-452f-a8f1-da5d5ada3c64 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:27.063191 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6336e6-b512-4f42-aa07-432d40bb1e7b None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:27.074409 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:27.096841 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4efec28-2555-4f09-9ed7-0198277f52e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:27.097801 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4efec28-2555-4f09-9ed7-0198277f52e3 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:27.098099 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4efec28-2555-4f09-9ed7-0198277f52e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:27.098378 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4efec28-2555-4f09-9ed7-0198277f52e3 None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:27.099218 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4efec28-2555-4f09-9ed7-0198277f52e3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:27.099525 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4efec28-2555-4f09-9ed7-0198277f52e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:27.101349 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4efec28-2555-4f09-9ed7-0198277f52e3 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:27.109563 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 280/577] 10.222.0.81 () {56 vars in 1103 bytes} [Wed Jul 20 22:26:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:27.110375 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4efec28-2555-4f09-9ed7-0198277f52e3 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:27.111473 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7485f66-907b-452f-a8f1-da5d5ada3c64 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:27.145223 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a363b86-a8c1-4326-861d-bb1b11417a14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:27.146054 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a363b86-a8c1-4326-861d-bb1b11417a14 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:27.146340 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a363b86-a8c1-4326-861d-bb1b11417a14 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:27.146618 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a363b86-a8c1-4326-861d-bb1b11417a14 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:27.156670 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a363b86-a8c1-4326-861d-bb1b11417a14 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 22:26:27.165563 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-1a363b86-a8c1-4326-861d-bb1b11417a14 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 22:26:27.165563 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:27.165563 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:27.165563 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:27.165563 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:27.165563 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:27.165563 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:27.165563 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:27.165563 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:27.165563 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:27.165563 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:27.165563 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:27.165563 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 22:26:27.165563 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 22:26:27.165563 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 22:26:27.165563 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 22:26:27.166640 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 22:26:27.166640 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 22:26:27.166640 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 22:26:27.166640 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 22:26:27.166640 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:27.166640 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:27.166640 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:27.166640 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:27.166640 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:27.166640 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:27.166640 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:27.166640 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:27.166640 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:27.166640 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:27.166640 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:27.166640 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:27.167684 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:27.167684 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:27.167684 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:27.167684 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:27.167684 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 22:26:27.167684 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 22:26:27.167684 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 22:26:27.167684 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 22:26:27.167684 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 22:26:27.167684 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 22:26:27.167684 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 22:26:27.167684 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:26:27.167684 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 298/578] 10.222.0.81 () {56 vars in 1102 bytes} [Wed Jul 20 22:26:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:27.167684 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a363b86-a8c1-4326-861d-bb1b11417a14 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:27.193725 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:27.214996 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4efec28-2555-4f09-9ed7-0198277f52e3 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:27.247765 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf65395e-d9ce-4f4b-bf43-0317e953c144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:27.248588 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf65395e-d9ce-4f4b-bf43-0317e953c144 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:27.248871 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf65395e-d9ce-4f4b-bf43-0317e953c144 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:27.249141 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf65395e-d9ce-4f4b-bf43-0317e953c144 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:27.256219 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf65395e-d9ce-4f4b-bf43-0317e953c144 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:27.257657 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf65395e-d9ce-4f4b-bf43-0317e953c144 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:27.271760 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a363b86-a8c1-4326-861d-bb1b11417a14 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:27.272697 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-bf65395e-d9ce-4f4b-bf43-0317e953c144 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 22:26:27.272697 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:27.272697 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:27.272697 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:27.272697 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:27.272697 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:27.272697 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:27.272697 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:27.272697 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:27.272697 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:27.272697 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:27.272697 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:27.272697 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 22:26:27.272697 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 22:26:27.272697 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 22:26:27.273616 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 22:26:27.273616 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:27.273616 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:27.273616 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:27.273616 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:27.273616 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:27.273616 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:27.273616 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:27.273616 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:27.273616 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:27.273616 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:27.273616 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:27.273616 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:27.273616 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:27.273616 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:27.273616 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:27.273616 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:27.274498 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 22:26:27.274498 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 22:26:27.274498 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 22:26:27.274498 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 22:26:27.274498 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 22:26:27.274498 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 22:26:27.274498 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 22:26:27.274498 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:26:27.274498 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 281/579] 10.222.0.81 () {56 vars in 1111 bytes} [Wed Jul 20 22:26:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:27.275035 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf65395e-d9ce-4f4b-bf43-0317e953c144 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:27.304447 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:27.305889 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4725c677-9159-455b-a241-e7aeca355f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:27.306541 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4725c677-9159-455b-a241-e7aeca355f6a None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:27.306822 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4725c677-9159-455b-a241-e7aeca355f6a None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:27.307132 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4725c677-9159-455b-a241-e7aeca355f6a None admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:27.307866 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4725c677-9159-455b-a241-e7aeca355f6a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:27.308169 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4725c677-9159-455b-a241-e7aeca355f6a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:27.309550 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4725c677-9159-455b-a241-e7aeca355f6a None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:27.321263 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 299/580] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Jul 20 22:26:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:27.322057 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4725c677-9159-455b-a241-e7aeca355f6a None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:27.373495 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf65395e-d9ce-4f4b-bf43-0317e953c144 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:27.401767 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:27.408421 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784954e3-599f-4a95-b720-e40c641d5b11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:27.409237 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784954e3-599f-4a95-b720-e40c641d5b11 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:27.409525 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784954e3-599f-4a95-b720-e40c641d5b11 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:27.409800 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784954e3-599f-4a95-b720-e40c641d5b11 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:27.416756 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784954e3-599f-4a95-b720-e40c641d5b11 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:27.418212 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784954e3-599f-4a95-b720-e40c641d5b11 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:27.420003 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4725c677-9159-455b-a241-e7aeca355f6a None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:27.432397 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-784954e3-599f-4a95-b720-e40c641d5b11 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 22:26:27.432397 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:27.432397 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:27.432397 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:27.432397 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:27.432397 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:27.432397 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:27.432397 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:27.432397 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:27.432397 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:27.432397 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:27.432397 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:27.432397 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 22:26:27.432397 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 22:26:27.432397 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 22:26:27.433368 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 22:26:27.433368 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:27.433368 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:27.433368 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:27.433368 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:27.433368 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:27.433368 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:27.433368 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:27.433368 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:27.433368 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:27.433368 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:27.433368 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:27.433368 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:27.433368 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:27.433368 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:27.433368 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:27.433368 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:27.434295 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 22:26:27.434295 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 22:26:27.434295 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 22:26:27.434295 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 22:26:27.434295 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 22:26:27.434295 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 22:26:27.434295 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 22:26:27.434295 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:26:27.438245 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 282/581] 10.222.0.81 () {56 vars in 1111 bytes} [Wed Jul 20 22:26:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:27.439087 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784954e3-599f-4a95-b720-e40c641d5b11 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:27.454271 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960f28fe-127f-4a80-8fd3-15f6ba003d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:27.455096 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960f28fe-127f-4a80-8fd3-15f6ba003d2e None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:27.455379 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960f28fe-127f-4a80-8fd3-15f6ba003d2e None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:27.455659 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960f28fe-127f-4a80-8fd3-15f6ba003d2e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:27.466138 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960f28fe-127f-4a80-8fd3-15f6ba003d2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:27.467553 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960f28fe-127f-4a80-8fd3-15f6ba003d2e None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:27.485107 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-960f28fe-127f-4a80-8fd3-15f6ba003d2e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:26:27.485107 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:27.485107 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:27.485107 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:27.485107 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:27.485107 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:27.485107 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:27.485107 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:27.485107 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:27.485107 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:27.485107 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:27.485107 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:27.485107 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 22:26:27.485107 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 22:26:27.485107 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 22:26:27.486132 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 22:26:27.486132 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:27.486132 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:27.486132 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:27.486132 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:27.486132 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:27.486132 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:27.486132 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:27.486132 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:27.486132 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:27.486132 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:27.486132 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:27.486132 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:27.486132 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:27.486132 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:27.486132 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:27.486132 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:27.487107 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 22:26:27.487107 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 22:26:27.487107 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:26:27.487107 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:26:27.487107 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 300/582] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Jul 20 22:26:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:27.487107 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960f28fe-127f-4a80-8fd3-15f6ba003d2e None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:27.534036 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:27.542814 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784954e3-599f-4a95-b720-e40c641d5b11 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:27.577398 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cd7aeb-6b20-4b73-acb6-dfb92f2bc051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:27.578069 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cd7aeb-6b20-4b73-acb6-dfb92f2bc051 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:27.578357 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cd7aeb-6b20-4b73-acb6-dfb92f2bc051 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:27.578663 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cd7aeb-6b20-4b73-acb6-dfb92f2bc051 None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:27.579438 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cd7aeb-6b20-4b73-acb6-dfb92f2bc051 None admin] RBAC: Adding query filter params (name=service) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:27.579752 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cd7aeb-6b20-4b73-acb6-dfb92f2bc051 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:27.581125 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cd7aeb-6b20-4b73-acb6-dfb92f2bc051 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:27.588890 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 283/583] 10.222.0.81 () {56 vars in 1103 bytes} [Wed Jul 20 22:26:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:27.589667 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cd7aeb-6b20-4b73-acb6-dfb92f2bc051 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:27.590182 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960f28fe-127f-4a80-8fd3-15f6ba003d2e None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:27.621131 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:27.625869 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79864bcb-6005-4133-97be-42c207845c9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:27.626527 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79864bcb-6005-4133-97be-42c207845c9f None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:27.626817 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79864bcb-6005-4133-97be-42c207845c9f None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:27.627124 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79864bcb-6005-4133-97be-42c207845c9f None admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:27.627909 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79864bcb-6005-4133-97be-42c207845c9f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:27.628222 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79864bcb-6005-4133-97be-42c207845c9f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:27.629659 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79864bcb-6005-4133-97be-42c207845c9f None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:27.643415 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 301/584] 10.222.0.81 () {56 vars in 1148 bytes} [Wed Jul 20 22:26:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:27.644202 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79864bcb-6005-4133-97be-42c207845c9f None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:27.692567 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cd7aeb-6b20-4b73-acb6-dfb92f2bc051 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:27.726319 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7493adc8-3520-449d-a383-7898aa7c4df4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:27.727088 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7493adc8-3520-449d-a383-7898aa7c4df4 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:27.727381 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7493adc8-3520-449d-a383-7898aa7c4df4 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:27.727663 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7493adc8-3520-449d-a383-7898aa7c4df4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:27.734152 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7493adc8-3520-449d-a383-7898aa7c4df4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:27.735633 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7493adc8-3520-449d-a383-7898aa7c4df4 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:27.742849 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79864bcb-6005-4133-97be-42c207845c9f None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:27.749994 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-7493adc8-3520-449d-a383-7898aa7c4df4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 22:26:27.749994 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:27.749994 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:27.749994 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:27.749994 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:27.749994 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:27.749994 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:27.749994 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:27.749994 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:27.749994 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:27.749994 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:27.749994 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:27.749994 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 22:26:27.749994 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 22:26:27.749994 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 22:26:27.750903 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 22:26:27.750903 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:27.750903 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:27.750903 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:27.750903 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:27.750903 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:27.750903 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:27.750903 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:27.750903 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:27.750903 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:27.750903 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:27.750903 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:27.750903 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:27.750903 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:27.750903 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:27.750903 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:27.750903 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:27.751820 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 22:26:27.751820 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 22:26:27.751820 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 22:26:27.751820 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 22:26:27.751820 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 22:26:27.751820 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 22:26:27.751820 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 22:26:27.751820 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:26:27.751820 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 284/585] 10.222.0.81 () {56 vars in 1111 bytes} [Wed Jul 20 22:26:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:27.752326 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7493adc8-3520-449d-a383-7898aa7c4df4 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:27.771076 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:27.781124 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631b171b-3504-4d33-b4f8-4747d6b6443b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:27.785388 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631b171b-3504-4d33-b4f8-4747d6b6443b None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:27.785388 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631b171b-3504-4d33-b4f8-4747d6b6443b None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:27.785388 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631b171b-3504-4d33-b4f8-4747d6b6443b None admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:27.785388 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631b171b-3504-4d33-b4f8-4747d6b6443b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:27.785388 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631b171b-3504-4d33-b4f8-4747d6b6443b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:27.785388 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631b171b-3504-4d33-b4f8-4747d6b6443b None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:27.796989 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 302/586] 10.222.0.81 () {56 vars in 1148 bytes} [Wed Jul 20 22:26:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:27.797774 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631b171b-3504-4d33-b4f8-4747d6b6443b None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:27.849976 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:27.856412 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7493adc8-3520-449d-a383-7898aa7c4df4 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:27.890986 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbb64e3-70d3-4562-80ef-fa160091849e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:27.891682 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbb64e3-70d3-4562-80ef-fa160091849e None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:27.891967 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbb64e3-70d3-4562-80ef-fa160091849e None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:27.892246 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbb64e3-70d3-4562-80ef-fa160091849e None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:27.892963 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbb64e3-70d3-4562-80ef-fa160091849e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:27.893274 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbb64e3-70d3-4562-80ef-fa160091849e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:27.894636 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbb64e3-70d3-4562-80ef-fa160091849e None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:27.908378 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 285/587] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Jul 20 22:26:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:27.908859 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631b171b-3504-4d33-b4f8-4747d6b6443b None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:27.909131 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbb64e3-70d3-4562-80ef-fa160091849e None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:27.946758 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350decbb-4e8d-45d9-a757-69b6457e4a9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:27.947592 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350decbb-4e8d-45d9-a757-69b6457e4a9c None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:27.947911 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350decbb-4e8d-45d9-a757-69b6457e4a9c None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:27.948188 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350decbb-4e8d-45d9-a757-69b6457e4a9c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:27.958808 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350decbb-4e8d-45d9-a757-69b6457e4a9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:27.960317 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350decbb-4e8d-45d9-a757-69b6457e4a9c None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:27.978287 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-350decbb-4e8d-45d9-a757-69b6457e4a9c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:26:27.978287 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:27.978287 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:27.978287 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:27.978287 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:27.978287 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:27.978287 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:27.978287 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:27.978287 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:27.978287 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:27.978287 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:27.978287 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:27.978287 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 22:26:27.978287 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 22:26:27.978287 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 22:26:27.979350 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 22:26:27.979350 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:27.979350 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:27.979350 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:27.979350 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:27.979350 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:27.979350 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:27.979350 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:27.979350 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:27.979350 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:27.979350 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:27.979350 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:27.979350 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:27.979350 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:27.979350 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:27.979350 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:27.979350 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:27.980457 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 22:26:27.980457 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 22:26:27.980457 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:26:27.980457 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:26:27.980457 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 303/588] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Jul 20 22:26:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:27.980457 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350decbb-4e8d-45d9-a757-69b6457e4a9c None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:27.987832 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:28.012566 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbb64e3-70d3-4562-80ef-fa160091849e None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:28.046737 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5654e974-fbca-4939-9e04-14b50a077e45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:28.047432 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5654e974-fbca-4939-9e04-14b50a077e45 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:28.047716 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5654e974-fbca-4939-9e04-14b50a077e45 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:28.047996 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5654e974-fbca-4939-9e04-14b50a077e45 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:28.048806 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5654e974-fbca-4939-9e04-14b50a077e45 None admin] RBAC: Adding query filter params (role.id=9b4a83e2eca44a8da9f356bc40f8788a, scope.project.id=385edf2e59e546d090ace36600e29032, user.id=70e97436a08543a180225561ef8bc817) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:28.049119 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5654e974-fbca-4939-9e04-14b50a077e45 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:28.050539 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5654e974-fbca-4939-9e04-14b50a077e45 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:28.056484 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 286/589] 10.222.0.81 () {56 vars in 1374 bytes} [Wed Jul 20 22:26:27 2022] GET /identity/v3/role_assignments?user.id=70e97436a08543a180225561ef8bc817&scope.project.id=385edf2e59e546d090ace36600e29032&role.id=9b4a83e2eca44a8da9f356bc40f8788a => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:28.057231 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5654e974-fbca-4939-9e04-14b50a077e45 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:28.081417 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350decbb-4e8d-45d9-a757-69b6457e4a9c None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:28.084031 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:28.119441 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b7f0e7-a79a-4332-a8cf-1799ddb6f3a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:28.120119 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b7f0e7-a79a-4332-a8cf-1799ddb6f3a2 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:28.120407 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b7f0e7-a79a-4332-a8cf-1799ddb6f3a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:28.120689 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b7f0e7-a79a-4332-a8cf-1799ddb6f3a2 None admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:28.121496 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b7f0e7-a79a-4332-a8cf-1799ddb6f3a2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:28.121810 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b7f0e7-a79a-4332-a8cf-1799ddb6f3a2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:28.123294 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b7f0e7-a79a-4332-a8cf-1799ddb6f3a2 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:28.136372 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 304/590] 10.222.0.81 () {56 vars in 1148 bytes} [Wed Jul 20 22:26:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:28.137244 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b7f0e7-a79a-4332-a8cf-1799ddb6f3a2 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:28.213169 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5654e974-fbca-4939-9e04-14b50a077e45 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:28.252152 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342a6864-cc70-4131-aa3c-ff0c3d628c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:28.252982 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342a6864-cc70-4131-aa3c-ff0c3d628c61 None admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:28.253270 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342a6864-cc70-4131-aa3c-ff0c3d628c61 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:28.253552 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342a6864-cc70-4131-aa3c-ff0c3d628c61 None admin] PATH_INFO: `/v3/projects/385edf2e59e546d090ace36600e29032/users/7829324db2574d38b18c749d0559177f/roles/9b4a83e2eca44a8da9f356bc40f8788a` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:28.262570 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:28.264683 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b7f0e7-a79a-4332-a8cf-1799ddb6f3a2 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:28.270099 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342a6864-cc70-4131-aa3c-ff0c3d628c61 None admin] RBAC: Authorizing `identity:create_grant(project_id=385edf2e59e546d090ace36600e29032, user_id=7829324db2574d38b18c749d0559177f, role_id=9b4a83e2eca44a8da9f356bc40f8788a)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:28.272574 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342a6864-cc70-4131-aa3c-ff0c3d628c61 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:28.299622 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2279a0-24b1-476d-9c29-2dbaa0d2fee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:28.300432 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2279a0-24b1-476d-9c29-2dbaa0d2fee1 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:28.300718 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2279a0-24b1-476d-9c29-2dbaa0d2fee1 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:28.301000 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2279a0-24b1-476d-9c29-2dbaa0d2fee1 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:28.309082 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2279a0-24b1-476d-9c29-2dbaa0d2fee1 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:28.310524 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2279a0-24b1-476d-9c29-2dbaa0d2fee1 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:28.325557 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-bd2279a0-24b1-476d-9c29-2dbaa0d2fee1 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 20 22:26:28.325557 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:28.325557 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:28.325557 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:28.325557 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:28.325557 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:28.325557 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:28.325557 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:28.325557 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:28.325557 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:28.325557 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:28.325557 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:28.325557 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 22:26:28.325557 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 22:26:28.325557 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 22:26:28.325557 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 22:26:28.326504 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:28.326504 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:28.326504 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 22:26:28.326504 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 22:26:28.326504 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 22:26:28.326504 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 22:26:28.326504 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:28.326504 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:28.326504 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:28.326504 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:28.326504 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:28.326504 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:28.326504 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:28.326504 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:28.326504 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:28.326504 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:28.326504 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:28.326504 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:28.327480 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:28.327480 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:28.327480 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 22:26:28.327480 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 22:26:28.327480 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 22:26:28.327480 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 22:26:28.327480 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 22:26:28.327480 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 22:26:28.327480 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 20 22:26:28.327480 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:26:28.327480 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 305/591] 10.222.0.81 () {56 vars in 1102 bytes} [Wed Jul 20 22:26:28 2022] GET /identity/v3/users/neutron => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:28.328070 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2279a0-24b1-476d-9c29-2dbaa0d2fee1 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:28.356949 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:28.365112 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 287/592] 10.222.0.81 () {58 vars in 1439 bytes} [Wed Jul 20 22:26:28 2022] PUT /identity/v3/projects/385edf2e59e546d090ace36600e29032/users/7829324db2574d38b18c749d0559177f/roles/9b4a83e2eca44a8da9f356bc40f8788a => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:26:28.365915 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342a6864-cc70-4131-aa3c-ff0c3d628c61 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:28.474758 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2279a0-24b1-476d-9c29-2dbaa0d2fee1 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:28.485086 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342a6864-cc70-4131-aa3c-ff0c3d628c61 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:28.533695 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9be30e6-4fef-4344-8b3e-cc017c2384e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:28.534300 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9be30e6-4fef-4344-8b3e-cc017c2384e7 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:28.534591 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9be30e6-4fef-4344-8b3e-cc017c2384e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:28.534868 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9be30e6-4fef-4344-8b3e-cc017c2384e7 None admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:28.536236 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9be30e6-4fef-4344-8b3e-cc017c2384e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:28.536554 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9be30e6-4fef-4344-8b3e-cc017c2384e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:28.537937 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9be30e6-4fef-4344-8b3e-cc017c2384e7 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:28.538044 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfda5a25-beff-4be6-933b-a40fa1e95169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:28.538860 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfda5a25-beff-4be6-933b-a40fa1e95169 None admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:28.539225 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfda5a25-beff-4be6-933b-a40fa1e95169 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:28.539642 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfda5a25-beff-4be6-933b-a40fa1e95169 None admin] PATH_INFO: `/v3/projects/385edf2e59e546d090ace36600e29032/users/9c44640ea0014426a9523bb1c457778b/roles/9b4a83e2eca44a8da9f356bc40f8788a` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:28.550743 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 306/593] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Jul 20 22:26:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:28.552119 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9be30e6-4fef-4344-8b3e-cc017c2384e7 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:28.562253 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfda5a25-beff-4be6-933b-a40fa1e95169 None admin] RBAC: Authorizing `identity:create_grant(project_id=385edf2e59e546d090ace36600e29032, user_id=9c44640ea0014426a9523bb1c457778b, role_id=9b4a83e2eca44a8da9f356bc40f8788a)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:28.563774 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfda5a25-beff-4be6-933b-a40fa1e95169 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:28.640984 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 288/594] 10.222.0.81 () {58 vars in 1439 bytes} [Wed Jul 20 22:26:28 2022] PUT /identity/v3/projects/385edf2e59e546d090ace36600e29032/users/9c44640ea0014426a9523bb1c457778b/roles/9b4a83e2eca44a8da9f356bc40f8788a => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:26:28.647039 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfda5a25-beff-4be6-933b-a40fa1e95169 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:28.661837 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:28.689080 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9be30e6-4fef-4344-8b3e-cc017c2384e7 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:28.732834 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:28.761692 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b05ed76-aef4-44f3-9533-4f40f72a2428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:28.762384 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b05ed76-aef4-44f3-9533-4f40f72a2428 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:28.762692 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b05ed76-aef4-44f3-9533-4f40f72a2428 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:28.762946 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b05ed76-aef4-44f3-9533-4f40f72a2428 None admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:28.763822 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b05ed76-aef4-44f3-9533-4f40f72a2428 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:28.764130 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b05ed76-aef4-44f3-9533-4f40f72a2428 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:28.765632 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b05ed76-aef4-44f3-9533-4f40f72a2428 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:28.803103 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 307/595] 10.222.0.81 () {56 vars in 1139 bytes} [Wed Jul 20 22:26:28 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:28.808063 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b05ed76-aef4-44f3-9533-4f40f72a2428 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:28.866433 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfda5a25-beff-4be6-933b-a40fa1e95169 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:28.907595 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6197a93b-9fc1-4979-8205-df31a4a25b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:28.908769 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6197a93b-9fc1-4979-8205-df31a4a25b8b None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:28.909616 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6197a93b-9fc1-4979-8205-df31a4a25b8b None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:28.910098 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6197a93b-9fc1-4979-8205-df31a4a25b8b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:28.919205 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6197a93b-9fc1-4979-8205-df31a4a25b8b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:28.921130 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6197a93b-9fc1-4979-8205-df31a4a25b8b None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:28.939928 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-6197a93b-9fc1-4979-8205-df31a4a25b8b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 20 22:26:28.939928 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:28.939928 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:28.939928 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:28.939928 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:28.939928 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:28.939928 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:28.939928 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:28.939928 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:28.939928 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:28.939928 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:28.939928 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:28.939928 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 22:26:28.939928 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 22:26:28.939928 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 22:26:28.939928 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 22:26:28.940904 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:28.940904 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:28.940904 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 22:26:28.940904 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 22:26:28.940904 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 22:26:28.940904 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 22:26:28.940904 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:28.940904 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:28.940904 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:28.940904 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:28.940904 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:28.940904 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:28.940904 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:28.940904 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:28.940904 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:28.940904 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:28.940904 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:28.940904 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:28.941816 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:28.941816 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:28.941816 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 22:26:28.941816 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 22:26:28.941816 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 22:26:28.941816 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 22:26:28.941816 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 22:26:28.941816 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 22:26:28.941816 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 20 22:26:28.941816 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:26:28.944722 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 289/596] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Jul 20 22:26:28 2022] GET /identity/v3/users/cinder => generated 84 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:28.946834 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b05ed76-aef4-44f3-9533-4f40f72a2428 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:28.956546 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6197a93b-9fc1-4979-8205-df31a4a25b8b None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:28.987965 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf66df5-eddd-4e92-a106-2837a26c76b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:28.988672 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf66df5-eddd-4e92-a106-2837a26c76b9 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:28.988957 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf66df5-eddd-4e92-a106-2837a26c76b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:28.989809 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf66df5-eddd-4e92-a106-2837a26c76b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:29.000099 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf66df5-eddd-4e92-a106-2837a26c76b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:29.001560 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf66df5-eddd-4e92-a106-2837a26c76b9 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:29.003414 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:29.023060 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-acf66df5-eddd-4e92-a106-2837a26c76b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:26:29.023060 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:29.023060 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:29.023060 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:29.023060 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:29.023060 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:29.023060 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:29.023060 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:29.023060 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:29.023060 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:29.023060 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:29.023060 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:29.023060 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 22:26:29.023060 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 22:26:29.023060 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 22:26:29.023974 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 22:26:29.023974 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:29.023974 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:29.023974 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:29.023974 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:29.023974 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:29.023974 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:29.023974 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:29.023974 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:29.023974 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:29.023974 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:29.023974 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:29.023974 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:29.023974 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:29.023974 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:29.023974 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:29.023974 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:29.025000 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 22:26:29.025000 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 22:26:29.025000 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:26:29.025000 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:26:29.025000 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 308/597] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Jul 20 22:26:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:29.027723 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf66df5-eddd-4e92-a106-2837a26c76b9 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:29.127413 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:29.130871 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6197a93b-9fc1-4979-8205-df31a4a25b8b None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:29.194690 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9fa391-9719-422e-a034-d36ef3eb7093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:29.195478 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9fa391-9719-422e-a034-d36ef3eb7093 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:29.195761 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9fa391-9719-422e-a034-d36ef3eb7093 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:29.196081 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9fa391-9719-422e-a034-d36ef3eb7093 None admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:29.196868 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9fa391-9719-422e-a034-d36ef3eb7093 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:29.197179 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9fa391-9719-422e-a034-d36ef3eb7093 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:29.198606 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9fa391-9719-422e-a034-d36ef3eb7093 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:29.232437 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf66df5-eddd-4e92-a106-2837a26c76b9 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:29.247616 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 290/598] 10.222.0.81 () {56 vars in 1137 bytes} [Wed Jul 20 22:26:28 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:29.248518 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9fa391-9719-422e-a034-d36ef3eb7093 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:29.340881 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cd52ed-7d58-4a50-93ec-9b9209772725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:29.341579 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cd52ed-7d58-4a50-93ec-9b9209772725 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:29.341825 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cd52ed-7d58-4a50-93ec-9b9209772725 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:29.342102 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cd52ed-7d58-4a50-93ec-9b9209772725 None admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:29.342829 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cd52ed-7d58-4a50-93ec-9b9209772725 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:29.343161 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cd52ed-7d58-4a50-93ec-9b9209772725 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:29.344528 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cd52ed-7d58-4a50-93ec-9b9209772725 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:29.369585 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 309/599] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Jul 20 22:26:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 22:26:29.371767 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cd52ed-7d58-4a50-93ec-9b9209772725 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:29.388872 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9fa391-9719-422e-a034-d36ef3eb7093 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:29.456521 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed819be-fc74-4d5d-8168-8353083f480c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:29.457909 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed819be-fc74-4d5d-8168-8353083f480c None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:29.458362 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed819be-fc74-4d5d-8168-8353083f480c None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:29.458650 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed819be-fc74-4d5d-8168-8353083f480c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:29.468630 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:29.472990 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed819be-fc74-4d5d-8168-8353083f480c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:29.475644 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed819be-fc74-4d5d-8168-8353083f480c None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:29.510959 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-fed819be-fc74-4d5d-8168-8353083f480c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:26:29.510959 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:29.510959 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:29.510959 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:29.510959 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:29.510959 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:29.510959 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:29.510959 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:29.510959 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:29.510959 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:29.510959 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:29.510959 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:29.510959 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 22:26:29.510959 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 22:26:29.510959 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 22:26:29.511930 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 22:26:29.511930 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:29.511930 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:29.511930 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:29.511930 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:29.511930 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:29.511930 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:29.511930 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:29.511930 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:29.511930 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:29.511930 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:29.511930 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:29.511930 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:29.511930 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:29.511930 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:29.511930 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:29.511930 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:29.512828 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 22:26:29.512828 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 22:26:29.512828 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:26:29.512828 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:26:29.517256 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 291/600] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Jul 20 22:26:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:29.524052 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed819be-fc74-4d5d-8168-8353083f480c None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:29.543435 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:29.563881 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cd52ed-7d58-4a50-93ec-9b9209772725 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:29.612156 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ae4670-6cf8-420d-b772-3f25ae08a248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:29.612982 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ae4670-6cf8-420d-b772-3f25ae08a248 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:29.613268 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ae4670-6cf8-420d-b772-3f25ae08a248 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:29.613545 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ae4670-6cf8-420d-b772-3f25ae08a248 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:29.621337 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ae4670-6cf8-420d-b772-3f25ae08a248 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:29.622792 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ae4670-6cf8-420d-b772-3f25ae08a248 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:29.638265 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-f8ae4670-6cf8-420d-b772-3f25ae08a248 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 22:26:29.638265 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:29.638265 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:29.638265 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:29.638265 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:29.638265 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:29.638265 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:29.638265 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:29.638265 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:29.638265 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:29.638265 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:29.638265 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:29.638265 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 22:26:29.638265 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 22:26:29.638265 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 22:26:29.639437 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 22:26:29.639437 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:29.639437 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:29.639437 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:29.639437 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:29.639437 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:29.639437 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:29.639437 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:29.639437 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:29.639437 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:29.639437 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:29.639437 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:29.639437 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:29.639437 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:29.639437 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:29.639437 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:29.639437 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:29.640526 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 22:26:29.640526 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 22:26:29.640526 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 22:26:29.640526 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 22:26:29.640526 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 22:26:29.640526 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 22:26:29.640526 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 22:26:29.640526 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:26:29.644338 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 310/601] 10.222.0.81 () {56 vars in 1111 bytes} [Wed Jul 20 22:26:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:29.646181 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ae4670-6cf8-420d-b772-3f25ae08a248 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:29.693137 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed819be-fc74-4d5d-8168-8353083f480c None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:29.758712 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4e8d56-d4b6-4742-935f-624fbab8d235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:29.759425 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4e8d56-d4b6-4742-935f-624fbab8d235 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:29.759728 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4e8d56-d4b6-4742-935f-624fbab8d235 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:29.759975 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4e8d56-d4b6-4742-935f-624fbab8d235 None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:29.760693 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4e8d56-d4b6-4742-935f-624fbab8d235 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:29.760996 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4e8d56-d4b6-4742-935f-624fbab8d235 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:29.762367 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4e8d56-d4b6-4742-935f-624fbab8d235 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:29.783347 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 292/602] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Jul 20 22:26:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:29.790648 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4e8d56-d4b6-4742-935f-624fbab8d235 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:29.823626 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ae4670-6cf8-420d-b772-3f25ae08a248 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:29.843765 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:29.884705 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c415f823-3617-4f37-b590-e10f1ec4ad4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:29.885381 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c415f823-3617-4f37-b590-e10f1ec4ad4d None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:29.885667 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c415f823-3617-4f37-b590-e10f1ec4ad4d None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:29.885946 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c415f823-3617-4f37-b590-e10f1ec4ad4d None admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:29.886734 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c415f823-3617-4f37-b590-e10f1ec4ad4d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:29.887062 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c415f823-3617-4f37-b590-e10f1ec4ad4d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:29.888500 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c415f823-3617-4f37-b590-e10f1ec4ad4d None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:29.903445 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:29.911109 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 311/603] 10.222.0.81 () {56 vars in 1148 bytes} [Wed Jul 20 22:26:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 22:26:29.920870 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c415f823-3617-4f37-b590-e10f1ec4ad4d None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:29.960671 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4e8d56-d4b6-4742-935f-624fbab8d235 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:30.025072 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5686604a-4ed7-4831-84a0-406128e11460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:30.025897 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5686604a-4ed7-4831-84a0-406128e11460 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:30.026181 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5686604a-4ed7-4831-84a0-406128e11460 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:30.026459 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5686604a-4ed7-4831-84a0-406128e11460 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:30.039390 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5686604a-4ed7-4831-84a0-406128e11460 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:30.041131 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5686604a-4ed7-4831-84a0-406128e11460 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:30.064479 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-5686604a-4ed7-4831-84a0-406128e11460 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 22:26:30.064479 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:30.064479 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:30.064479 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:30.064479 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:30.064479 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:30.064479 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:30.064479 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:30.064479 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:30.064479 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:30.064479 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:30.064479 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:30.064479 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 22:26:30.064479 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 22:26:30.064479 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 22:26:30.066624 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 22:26:30.066624 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:30.066624 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:30.066624 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:30.066624 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:30.066624 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:30.066624 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:30.066624 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:30.066624 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:30.066624 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:30.066624 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:30.066624 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:30.066624 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:30.066624 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:30.066624 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:30.066624 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:30.066624 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:30.069365 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 22:26:30.069365 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 22:26:30.069365 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 22:26:30.069365 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 22:26:30.069365 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 22:26:30.069365 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 22:26:30.069365 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 22:26:30.069365 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:26:30.069365 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 293/604] 10.222.0.81 () {56 vars in 1111 bytes} [Wed Jul 20 22:26:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:30.076550 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5686604a-4ed7-4831-84a0-406128e11460 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:30.099102 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c415f823-3617-4f37-b590-e10f1ec4ad4d None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:30.163433 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adac410d-5290-470b-b469-10a7964c391c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:30.164250 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adac410d-5290-470b-b469-10a7964c391c None admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:30.164531 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adac410d-5290-470b-b469-10a7964c391c None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:30.165067 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adac410d-5290-470b-b469-10a7964c391c None admin] PATH_INFO: `/v3/projects/385edf2e59e546d090ace36600e29032/users/333c2b1b11cc4c478abfd965a72e9d66/roles/9b4a83e2eca44a8da9f356bc40f8788a` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:30.206392 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adac410d-5290-470b-b469-10a7964c391c None admin] RBAC: Authorizing `identity:create_grant(project_id=385edf2e59e546d090ace36600e29032, user_id=333c2b1b11cc4c478abfd965a72e9d66, role_id=9b4a83e2eca44a8da9f356bc40f8788a)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:30.208445 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adac410d-5290-470b-b469-10a7964c391c None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:30.250173 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5686604a-4ed7-4831-84a0-406128e11460 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:30.278840 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:30.287795 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c52591a-4388-459d-aac0-32dbc8828607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:30.288478 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c52591a-4388-459d-aac0-32dbc8828607 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:30.288763 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c52591a-4388-459d-aac0-32dbc8828607 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:30.289099 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c52591a-4388-459d-aac0-32dbc8828607 None admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:30.290149 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c52591a-4388-459d-aac0-32dbc8828607 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:30.290460 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c52591a-4388-459d-aac0-32dbc8828607 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:30.291927 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c52591a-4388-459d-aac0-32dbc8828607 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:30.305599 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 294/605] 10.222.0.81 () {56 vars in 1148 bytes} [Wed Jul 20 22:26:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 22:26:30.311056 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c52591a-4388-459d-aac0-32dbc8828607 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:30.330727 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 312/606] 10.222.0.81 () {58 vars in 1439 bytes} [Wed Jul 20 22:26:29 2022] PUT /identity/v3/projects/385edf2e59e546d090ace36600e29032/users/333c2b1b11cc4c478abfd965a72e9d66/roles/9b4a83e2eca44a8da9f356bc40f8788a => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 22:26:30.450073 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c52591a-4388-459d-aac0-32dbc8828607 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:30.501850 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32be1b41-1ed4-4827-a972-fd9c9c84203b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:30.502895 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32be1b41-1ed4-4827-a972-fd9c9c84203b None admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:30.503204 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32be1b41-1ed4-4827-a972-fd9c9c84203b None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:30.503482 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32be1b41-1ed4-4827-a972-fd9c9c84203b None admin] PATH_INFO: `/v3/projects/385edf2e59e546d090ace36600e29032/users/ecf4db289fba4e4a806198c62d31f7c4/roles/9b4a83e2eca44a8da9f356bc40f8788a` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:30.506287 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251bca68-d57b-468a-bce5-708db37cb764 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:30.506811 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251bca68-d57b-468a-bce5-708db37cb764 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:30.506981 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251bca68-d57b-468a-bce5-708db37cb764 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:30.511218 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 313/607] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:26:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:26:30.516483 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901399b4-e078-4df1-b7bb-9fff8b1054a7 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:30.516772 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901399b4-e078-4df1-b7bb-9fff8b1054a7 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:30.517112 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901399b4-e078-4df1-b7bb-9fff8b1054a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:30.523326 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32be1b41-1ed4-4827-a972-fd9c9c84203b None admin] RBAC: Authorizing `identity:create_grant(project_id=385edf2e59e546d090ace36600e29032, user_id=ecf4db289fba4e4a806198c62d31f7c4, role_id=9b4a83e2eca44a8da9f356bc40f8788a)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:30.525266 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32be1b41-1ed4-4827-a972-fd9c9c84203b None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:30.537868 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-901399b4-e078-4df1-b7bb-9fff8b1054a7 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:26:30.539213 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-901399b4-e078-4df1-b7bb-9fff8b1054a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:26:30.558155 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:30.584332 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 295/608] 10.222.0.81 () {58 vars in 1439 bytes} [Wed Jul 20 22:26:30 2022] PUT /identity/v3/projects/385edf2e59e546d090ace36600e29032/users/ecf4db289fba4e4a806198c62d31f7c4/roles/9b4a83e2eca44a8da9f356bc40f8788a => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:26:30.586751 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd76adc5-3dfa-4d4a-9a08-dcae6ec3ab92 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:30.587649 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd76adc5-3dfa-4d4a-9a08-dcae6ec3ab92 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:30.588170 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd76adc5-3dfa-4d4a-9a08-dcae6ec3ab92 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:30.590507 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 296/609] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:26:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:26:30.596672 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472a8701-cfd7-4d63-9264-f8366e7375d6 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:30.597011 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472a8701-cfd7-4d63-9264-f8366e7375d6 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:30.597245 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472a8701-cfd7-4d63-9264-f8366e7375d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:30.615650 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-472a8701-cfd7-4d63-9264-f8366e7375d6 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:26:30.616910 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-472a8701-cfd7-4d63-9264-f8366e7375d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:26:30.618665 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 314/610] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:26:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 22:26:30.652032 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:30.660475 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901399b4-e078-4df1-b7bb-9fff8b1054a7 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:30.702648 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 297/611] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:26:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 22:26:30.766578 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472a8701-cfd7-4d63-9264-f8366e7375d6 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:30.814317 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901399b4-e078-4df1-b7bb-9fff8b1054a7 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:30.860672 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f34ddf-5505-49ba-9751-124e42adf279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:30.861949 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f34ddf-5505-49ba-9751-124e42adf279 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:30.862237 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f34ddf-5505-49ba-9751-124e42adf279 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:30.862515 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f34ddf-5505-49ba-9751-124e42adf279 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:30.878131 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f34ddf-5505-49ba-9751-124e42adf279 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 22:26:30.889487 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-27f34ddf-5505-49ba-9751-124e42adf279 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 22:26:30.889487 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:30.889487 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:30.889487 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:30.889487 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:30.889487 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:30.889487 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:30.889487 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:30.889487 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:30.889487 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:30.889487 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:30.889487 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:30.889487 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 22:26:30.889487 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 22:26:30.889487 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 22:26:30.889487 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 22:26:30.890445 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 22:26:30.890445 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 22:26:30.890445 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 22:26:30.890445 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 22:26:30.890445 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:30.890445 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:30.890445 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:30.890445 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:30.890445 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:30.890445 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:30.890445 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:30.890445 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:30.890445 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:30.890445 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:30.890445 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:30.890445 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:30.891343 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:30.891343 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:30.891343 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:30.891343 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:30.891343 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 22:26:30.891343 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 22:26:30.891343 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 22:26:30.891343 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 22:26:30.891343 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 22:26:30.891343 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 22:26:30.891343 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 22:26:30.891343 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:26:30.891343 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 315/612] 10.222.0.81 () {56 vars in 1102 bytes} [Wed Jul 20 22:26:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 22:26:30.904044 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f34ddf-5505-49ba-9751-124e42adf279 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:30.923212 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472a8701-cfd7-4d63-9264-f8366e7375d6 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:30.930274 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:30.967869 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399ae535-c0b0-42c9-8eeb-142eb5e4bbec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:30.968619 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399ae535-c0b0-42c9-8eeb-142eb5e4bbec None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:30.968908 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399ae535-c0b0-42c9-8eeb-142eb5e4bbec None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:30.969213 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399ae535-c0b0-42c9-8eeb-142eb5e4bbec None admin] PATH_INFO: `/v3/roles/service` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:30.990182 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399ae535-c0b0-42c9-8eeb-142eb5e4bbec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 22:26:31.004348 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-399ae535-c0b0-42c9-8eeb-142eb5e4bbec None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 22:26:31.004348 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:31.004348 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:31.004348 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:31.004348 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:31.004348 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:31.004348 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:31.004348 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:31.004348 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:31.004348 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:31.004348 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:31.004348 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:31.004348 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 22:26:31.004348 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 22:26:31.004348 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 22:26:31.004348 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 22:26:31.006494 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 22:26:31.006494 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 22:26:31.006494 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 22:26:31.006494 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 22:26:31.006494 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:31.006494 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:31.006494 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:31.006494 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:31.006494 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:31.006494 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:31.006494 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:31.006494 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:31.006494 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:31.006494 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:31.006494 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:31.006494 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:31.008512 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:31.008512 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:31.008512 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:31.008512 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:31.008512 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 22:26:31.008512 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 22:26:31.008512 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 22:26:31.008512 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 22:26:31.008512 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 22:26:31.008512 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 22:26:31.008512 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 22:26:31.008512 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:26:31.008512 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 298/613] 10.222.0.81 () {56 vars in 1102 bytes} [Wed Jul 20 22:26:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 22:26:31.016335 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399ae535-c0b0-42c9-8eeb-142eb5e4bbec None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:31.050731 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:31.059128 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f34ddf-5505-49ba-9751-124e42adf279 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:31.110000 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae52a334-ded2-434f-b6f1-7023e94f2fa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:31.110677 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae52a334-ded2-434f-b6f1-7023e94f2fa6 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:31.110963 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae52a334-ded2-434f-b6f1-7023e94f2fa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:31.111274 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae52a334-ded2-434f-b6f1-7023e94f2fa6 None admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:31.112039 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae52a334-ded2-434f-b6f1-7023e94f2fa6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:31.112345 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae52a334-ded2-434f-b6f1-7023e94f2fa6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:31.113746 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae52a334-ded2-434f-b6f1-7023e94f2fa6 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:31.123612 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 316/614] 10.222.0.81 () {56 vars in 1103 bytes} [Wed Jul 20 22:26:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:31.128874 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae52a334-ded2-434f-b6f1-7023e94f2fa6 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:31.161535 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399ae535-c0b0-42c9-8eeb-142eb5e4bbec None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:31.225694 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6eb1aef-f953-4413-ae50-08f2001a494c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:31.226307 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eb1aef-f953-4413-ae50-08f2001a494c None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:31.226599 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eb1aef-f953-4413-ae50-08f2001a494c None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:31.226885 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eb1aef-f953-4413-ae50-08f2001a494c None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:31.227667 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6eb1aef-f953-4413-ae50-08f2001a494c None admin] RBAC: Adding query filter params (name=service) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:31.227979 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6eb1aef-f953-4413-ae50-08f2001a494c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:31.229360 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6eb1aef-f953-4413-ae50-08f2001a494c None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:31.246775 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 299/615] 10.222.0.81 () {56 vars in 1103 bytes} [Wed Jul 20 22:26:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:31.260400 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6eb1aef-f953-4413-ae50-08f2001a494c None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:31.264967 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:31.291629 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae52a334-ded2-434f-b6f1-7023e94f2fa6 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:31.334848 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f821465-b5b2-4f51-ab1d-0b115480e1f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:31.335733 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f821465-b5b2-4f51-ab1d-0b115480e1f0 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:31.336018 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f821465-b5b2-4f51-ab1d-0b115480e1f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:31.336295 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f821465-b5b2-4f51-ab1d-0b115480e1f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:31.349449 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f821465-b5b2-4f51-ab1d-0b115480e1f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:31.351038 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f821465-b5b2-4f51-ab1d-0b115480e1f0 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:31.372731 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-2f821465-b5b2-4f51-ab1d-0b115480e1f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:26:31.372731 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:31.372731 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:31.372731 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:31.372731 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:31.372731 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:31.372731 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:31.372731 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:31.372731 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:31.372731 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:31.372731 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:31.372731 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:31.372731 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 22:26:31.372731 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 22:26:31.372731 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 22:26:31.373676 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 22:26:31.373676 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:31.373676 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:31.373676 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:31.373676 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:31.373676 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:31.373676 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:31.373676 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:31.373676 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:31.373676 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:31.373676 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:31.373676 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:31.373676 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:31.373676 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:31.373676 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:31.373676 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:31.373676 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:31.374540 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 22:26:31.374540 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 22:26:31.374540 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:26:31.374540 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:26:31.374540 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 317/616] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Jul 20 22:26:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:31.375694 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d164da6f-5e8a-461a-8e48-2c39b108bb88 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:31.375979 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d164da6f-5e8a-461a-8e48-2c39b108bb88 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:31.376315 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d164da6f-5e8a-461a-8e48-2c39b108bb88 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:31.377828 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 318/617] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:26:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:26:31.383357 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d164da6f-5e8a-461a-8e48-2c39b108bb88 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:31.408623 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:31.418918 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6eb1aef-f953-4413-ae50-08f2001a494c None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:31.456386 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da828414-f405-4a9a-977d-a11159c5268c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:31.457208 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da828414-f405-4a9a-977d-a11159c5268c None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:31.457520 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da828414-f405-4a9a-977d-a11159c5268c None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:31.457807 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da828414-f405-4a9a-977d-a11159c5268c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:31.471867 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da828414-f405-4a9a-977d-a11159c5268c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:31.473452 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da828414-f405-4a9a-977d-a11159c5268c None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:31.492736 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-da828414-f405-4a9a-977d-a11159c5268c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:26:31.492736 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:31.492736 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:31.492736 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:31.492736 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:31.492736 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:31.492736 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:31.492736 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:31.492736 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:31.492736 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:31.492736 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:31.492736 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:31.492736 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 22:26:31.492736 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 22:26:31.492736 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 22:26:31.494486 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 22:26:31.494486 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:31.494486 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:31.494486 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:31.494486 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:31.494486 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:31.494486 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:31.494486 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:31.494486 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:31.494486 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:31.494486 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:31.494486 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:31.494486 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:31.494486 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:31.494486 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:31.494486 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:31.494486 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:31.496319 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 22:26:31.496319 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 22:26:31.496319 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:26:31.496319 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:26:31.496720 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d164da6f-5e8a-461a-8e48-2c39b108bb88 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:31.496843 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 300/618] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Jul 20 22:26:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 22:26:31.496843 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d92200-ab68-42e5-9372-1b0a4a5552c2 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:31.496843 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d92200-ab68-42e5-9372-1b0a4a5552c2 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:31.496843 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d92200-ab68-42e5-9372-1b0a4a5552c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:31.514399 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-f1d92200-ab68-42e5-9372-1b0a4a5552c2 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:26:31.515752 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-f1d92200-ab68-42e5-9372-1b0a4a5552c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:26:31.528663 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:31.537314 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf534a8c-ccfd-4553-b2e8-26d7c73d971f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:31.540958 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf534a8c-ccfd-4553-b2e8-26d7c73d971f None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:31.540958 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf534a8c-ccfd-4553-b2e8-26d7c73d971f None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:31.540958 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf534a8c-ccfd-4553-b2e8-26d7c73d971f None admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:31.540958 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf534a8c-ccfd-4553-b2e8-26d7c73d971f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:31.540958 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf534a8c-ccfd-4553-b2e8-26d7c73d971f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:31.541726 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf534a8c-ccfd-4553-b2e8-26d7c73d971f None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:31.554235 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 319/619] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Jul 20 22:26:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:31.555300 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf534a8c-ccfd-4553-b2e8-26d7c73d971f None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:31.596692 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 301/620] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:26:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 22:26:31.597783 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d92200-ab68-42e5-9372-1b0a4a5552c2 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:31.673806 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:31.680188 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf534a8c-ccfd-4553-b2e8-26d7c73d971f None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:31.706962 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d92200-ab68-42e5-9372-1b0a4a5552c2 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:31.728004 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9103862a-526a-48c3-a47d-6ee79ffca4fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:31.728805 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9103862a-526a-48c3-a47d-6ee79ffca4fc None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:31.729195 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9103862a-526a-48c3-a47d-6ee79ffca4fc None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:31.729579 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9103862a-526a-48c3-a47d-6ee79ffca4fc None admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:31.730932 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9103862a-526a-48c3-a47d-6ee79ffca4fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:31.731367 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9103862a-526a-48c3-a47d-6ee79ffca4fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:31.733127 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9103862a-526a-48c3-a47d-6ee79ffca4fc None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:31.745767 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ab9867-57f4-4c72-a8fa-933da522ac9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:31.746761 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ab9867-57f4-4c72-a8fa-933da522ac9b None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:31.747289 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ab9867-57f4-4c72-a8fa-933da522ac9b None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:31.747748 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ab9867-57f4-4c72-a8fa-933da522ac9b None admin] PATH_INFO: `/v3/users/swift` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:31.752970 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 320/621] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Jul 20 22:26:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:31.752970 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9103862a-526a-48c3-a47d-6ee79ffca4fc None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:31.761114 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ab9867-57f4-4c72-a8fa-933da522ac9b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:31.762874 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ab9867-57f4-4c72-a8fa-933da522ac9b None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:31.779601 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-49ab9867-57f4-4c72-a8fa-933da522ac9b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 20 22:26:31.779601 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:31.779601 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:31.779601 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:31.779601 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:31.779601 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:31.779601 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:31.779601 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:31.779601 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:31.779601 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:31.779601 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:31.779601 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:31.779601 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 22:26:31.779601 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 22:26:31.779601 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 22:26:31.779601 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 22:26:31.780720 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:31.780720 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:31.780720 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 22:26:31.780720 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 22:26:31.780720 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 22:26:31.780720 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 22:26:31.780720 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:31.780720 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:31.780720 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:31.780720 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:31.780720 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:31.780720 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:31.780720 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:31.780720 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:31.780720 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:31.780720 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:31.780720 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:31.780720 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:31.781682 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:31.781682 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:31.781682 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 22:26:31.781682 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 22:26:31.781682 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 22:26:31.781682 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 22:26:31.781682 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 22:26:31.781682 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 22:26:31.781682 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 20 22:26:31.781682 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:26:31.784065 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 302/622] 10.222.0.81 () {56 vars in 1096 bytes} [Wed Jul 20 22:26:31 2022] GET /identity/v3/users/swift => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:31.785021 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ab9867-57f4-4c72-a8fa-933da522ac9b None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:31.811443 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:31.880354 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9103862a-526a-48c3-a47d-6ee79ffca4fc None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:31.893599 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ab9867-57f4-4c72-a8fa-933da522ac9b None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:31.929802 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76f5954-0e73-44f5-be38-164b5b5b8cea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:31.930505 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22eab42f-b2f5-43d5-b65e-e8cce745b884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:31.931076 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76f5954-0e73-44f5-be38-164b5b5b8cea None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:31.931512 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76f5954-0e73-44f5-be38-164b5b5b8cea None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:31.931652 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eab42f-b2f5-43d5-b65e-e8cce745b884 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:31.932040 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eab42f-b2f5-43d5-b65e-e8cce745b884 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:31.932223 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76f5954-0e73-44f5-be38-164b5b5b8cea None admin] PATH_INFO: `/v3/roles/service` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:31.932501 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eab42f-b2f5-43d5-b65e-e8cce745b884 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:31.935704 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:31.952621 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22eab42f-b2f5-43d5-b65e-e8cce745b884 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:31.952621 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22eab42f-b2f5-43d5-b65e-e8cce745b884 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:31.952905 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76f5954-0e73-44f5-be38-164b5b5b8cea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 22:26:31.962239 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-e76f5954-0e73-44f5-be38-164b5b5b8cea None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 22:26:31.962239 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:31.962239 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:31.962239 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:31.962239 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:31.962239 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:31.962239 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:31.962239 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:31.962239 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:31.962239 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:31.962239 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:31.962239 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:31.962239 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 22:26:31.962239 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 22:26:31.962239 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 22:26:31.962239 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 22:26:31.963427 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 22:26:31.963427 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 22:26:31.963427 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 22:26:31.963427 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 22:26:31.963427 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:31.963427 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:31.963427 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:31.963427 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:31.963427 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:31.963427 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:31.963427 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:31.963427 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:31.963427 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:31.963427 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:31.963427 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:31.963427 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:31.964445 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:31.964445 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:31.964445 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:31.964445 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:31.964445 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 22:26:31.964445 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 22:26:31.964445 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 22:26:31.964445 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 22:26:31.964445 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 22:26:31.964445 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 22:26:31.964445 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 22:26:31.964445 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:26:31.968792 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-22eab42f-b2f5-43d5-b65e-e8cce745b884 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 20 22:26:31.968792 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:31.968792 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:31.968792 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:31.968792 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:31.968792 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:31.968792 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:31.968792 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:31.968792 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:31.968792 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:31.968792 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:31.968792 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:31.968792 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 22:26:31.968792 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 22:26:31.968792 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 22:26:31.968792 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 22:26:31.969733 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:31.969733 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:31.969733 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 22:26:31.969733 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 22:26:31.969733 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 22:26:31.969733 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 22:26:31.969733 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:31.969733 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:31.969733 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:31.969733 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:31.969733 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:31.969733 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:31.969733 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:31.969733 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:31.969733 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:31.969733 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:31.969733 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:31.969733 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:31.970649 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:31.970649 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:31.970649 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 22:26:31.970649 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 22:26:31.970649 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 22:26:31.970649 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 22:26:31.970649 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 22:26:31.970649 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 22:26:31.970649 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 20 22:26:31.970649 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:26:31.971219 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 321/623] 10.222.0.81 () {56 vars in 1102 bytes} [Wed Jul 20 22:26:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:31.971290 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 303/624] 10.222.0.81 () {56 vars in 1093 bytes} [Wed Jul 20 22:26:31 2022] GET /identity/v3/users/nova => generated 82 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:31.971357 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76f5954-0e73-44f5-be38-164b5b5b8cea None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:31.971778 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22eab42f-b2f5-43d5-b65e-e8cce745b884 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:32.094474 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22eab42f-b2f5-43d5-b65e-e8cce745b884 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:32.097976 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76f5954-0e73-44f5-be38-164b5b5b8cea None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:32.108829 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:32.131735 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecdb5d1-8caa-4756-977a-e62821e49b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:32.132652 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecdb5d1-8caa-4756-977a-e62821e49b58 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:32.133089 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecdb5d1-8caa-4756-977a-e62821e49b58 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:32.133545 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecdb5d1-8caa-4756-977a-e62821e49b58 None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:32.134439 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecdb5d1-8caa-4756-977a-e62821e49b58 None admin] RBAC: Adding query filter params (name=service) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:32.134959 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecdb5d1-8caa-4756-977a-e62821e49b58 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:32.136587 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecdb5d1-8caa-4756-977a-e62821e49b58 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:32.139661 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef2f02b-1198-4fc7-a1be-e19a73c3fa62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:32.141900 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef2f02b-1198-4fc7-a1be-e19a73c3fa62 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:32.143417 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef2f02b-1198-4fc7-a1be-e19a73c3fa62 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:32.143799 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef2f02b-1198-4fc7-a1be-e19a73c3fa62 None admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:32.145335 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef2f02b-1198-4fc7-a1be-e19a73c3fa62 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:32.146092 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef2f02b-1198-4fc7-a1be-e19a73c3fa62 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:32.152730 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef2f02b-1198-4fc7-a1be-e19a73c3fa62 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:32.153173 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 304/625] 10.222.0.81 () {56 vars in 1103 bytes} [Wed Jul 20 22:26:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:32.154186 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecdb5d1-8caa-4756-977a-e62821e49b58 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:32.174345 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 322/626] 10.222.0.81 () {56 vars in 1135 bytes} [Wed Jul 20 22:26:31 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:32.174345 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef2f02b-1198-4fc7-a1be-e19a73c3fa62 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:32.227748 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:32.277422 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecdb5d1-8caa-4756-977a-e62821e49b58 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:32.313290 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef2f02b-1198-4fc7-a1be-e19a73c3fa62 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:32.321161 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcc3b22-04b9-40c1-97eb-a90de0d4f610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:32.322029 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcc3b22-04b9-40c1-97eb-a90de0d4f610 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:32.322495 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcc3b22-04b9-40c1-97eb-a90de0d4f610 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:32.322939 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcc3b22-04b9-40c1-97eb-a90de0d4f610 None admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:32.324000 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcc3b22-04b9-40c1-97eb-a90de0d4f610 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:32.324463 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcc3b22-04b9-40c1-97eb-a90de0d4f610 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:32.326121 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcc3b22-04b9-40c1-97eb-a90de0d4f610 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:32.349611 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 305/627] 10.222.0.81 () {56 vars in 1133 bytes} [Wed Jul 20 22:26:32 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:32.350843 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcc3b22-04b9-40c1-97eb-a90de0d4f610 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:32.353170 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74942e4a-4cf3-4bd9-9530-a23954280cad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:32.354094 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74942e4a-4cf3-4bd9-9530-a23954280cad None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:32.354389 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74942e4a-4cf3-4bd9-9530-a23954280cad None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:32.354668 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74942e4a-4cf3-4bd9-9530-a23954280cad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:32.366520 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74942e4a-4cf3-4bd9-9530-a23954280cad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:32.368194 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74942e4a-4cf3-4bd9-9530-a23954280cad None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:32.388670 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-74942e4a-4cf3-4bd9-9530-a23954280cad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:26:32.388670 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:32.388670 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:32.388670 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:32.388670 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:32.388670 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:32.388670 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:32.388670 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:32.388670 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:32.388670 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:32.388670 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:32.388670 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:32.388670 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 22:26:32.388670 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 22:26:32.388670 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 22:26:32.389682 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 22:26:32.389682 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:32.389682 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:32.389682 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:32.389682 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:32.389682 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:32.389682 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:32.389682 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:32.389682 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:32.389682 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:32.389682 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:32.389682 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:32.389682 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:32.389682 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:32.389682 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:32.389682 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:32.389682 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:32.390605 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 22:26:32.390605 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 22:26:32.390605 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:26:32.390605 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:26:32.394547 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 323/628] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Jul 20 22:26:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:32.395524 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74942e4a-4cf3-4bd9-9530-a23954280cad None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:32.417322 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:32.463915 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcc3b22-04b9-40c1-97eb-a90de0d4f610 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:32.480388 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:32.506391 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564325b3-7ae7-4e07-96b6-01ea9fa8bae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:32.506881 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74942e4a-4cf3-4bd9-9530-a23954280cad None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:32.507870 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564325b3-7ae7-4e07-96b6-01ea9fa8bae2 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:32.508630 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564325b3-7ae7-4e07-96b6-01ea9fa8bae2 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:32.510031 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564325b3-7ae7-4e07-96b6-01ea9fa8bae2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:32.524167 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564325b3-7ae7-4e07-96b6-01ea9fa8bae2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:32.526380 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564325b3-7ae7-4e07-96b6-01ea9fa8bae2 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:32.545540 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7215247-2b48-4c6d-aaef-c886529f3d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:32.546213 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7215247-2b48-4c6d-aaef-c886529f3d31 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:32.546477 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7215247-2b48-4c6d-aaef-c886529f3d31 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:32.546760 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7215247-2b48-4c6d-aaef-c886529f3d31 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:32.553790 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-564325b3-7ae7-4e07-96b6-01ea9fa8bae2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:26:32.553790 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:32.553790 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:32.553790 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:32.553790 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:32.553790 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:32.553790 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:32.553790 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:32.553790 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:32.553790 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:32.553790 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:32.553790 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:32.553790 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 22:26:32.553790 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 22:26:32.553790 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 22:26:32.554713 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 22:26:32.554713 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:32.554713 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:32.554713 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:32.554713 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:32.554713 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:32.554713 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:32.554713 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:32.554713 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:32.554713 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:32.554713 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:32.554713 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:32.554713 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:32.554713 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:32.554713 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:32.554713 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:32.554713 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:32.556948 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 22:26:32.556948 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 22:26:32.556948 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:26:32.556948 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:26:32.564285 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 306/629] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Jul 20 22:26:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:32.564285 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564325b3-7ae7-4e07-96b6-01ea9fa8bae2 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:32.633026 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7215247-2b48-4c6d-aaef-c886529f3d31 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:32.634633 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7215247-2b48-4c6d-aaef-c886529f3d31 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:32.654556 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:32.672165 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564325b3-7ae7-4e07-96b6-01ea9fa8bae2 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:32.708927 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1df39a3-36e3-4527-83b0-9e401f8460e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:32.709597 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1df39a3-36e3-4527-83b0-9e401f8460e3 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:32.709884 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1df39a3-36e3-4527-83b0-9e401f8460e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:32.710161 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1df39a3-36e3-4527-83b0-9e401f8460e3 None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:32.710881 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1df39a3-36e3-4527-83b0-9e401f8460e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:32.711242 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1df39a3-36e3-4527-83b0-9e401f8460e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:32.712618 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1df39a3-36e3-4527-83b0-9e401f8460e3 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:32.728893 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 307/630] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Jul 20 22:26:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:32.728893 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1df39a3-36e3-4527-83b0-9e401f8460e3 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:32.753678 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 324/631] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Jul 20 22:26:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 22:26:32.753678 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099de95b-1fa2-4401-9966-e841d7812975 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:32.753678 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099de95b-1fa2-4401-9966-e841d7812975 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:32.753678 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099de95b-1fa2-4401-9966-e841d7812975 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:32.754774 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:32.755181 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 325/632] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:26:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:26:32.756572 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099de95b-1fa2-4401-9966-e841d7812975 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:32.837235 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1df39a3-36e3-4527-83b0-9e401f8460e3 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:32.873092 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f237a31-b750-4ee4-812b-058f05a894d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:32.873794 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f237a31-b750-4ee4-812b-058f05a894d5 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:32.874135 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f237a31-b750-4ee4-812b-058f05a894d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:32.874354 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f237a31-b750-4ee4-812b-058f05a894d5 None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:32.875096 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f237a31-b750-4ee4-812b-058f05a894d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:32.875406 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f237a31-b750-4ee4-812b-058f05a894d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:32.875667 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099de95b-1fa2-4401-9966-e841d7812975 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:32.876783 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f237a31-b750-4ee4-812b-058f05a894d5 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:32.893745 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 308/633] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Jul 20 22:26:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:32.893745 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f237a31-b750-4ee4-812b-058f05a894d5 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:32.899650 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:32.917848 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec18924-057b-42ae-a260-7a871adf7f38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:32.918652 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec18924-057b-42ae-a260-7a871adf7f38 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:32.918942 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec18924-057b-42ae-a260-7a871adf7f38 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:32.919268 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec18924-057b-42ae-a260-7a871adf7f38 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:32.926417 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec18924-057b-42ae-a260-7a871adf7f38 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:32.928123 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec18924-057b-42ae-a260-7a871adf7f38 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:32.943517 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-aec18924-057b-42ae-a260-7a871adf7f38 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 20 22:26:32.943517 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:32.943517 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:32.943517 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:32.943517 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:32.943517 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:32.943517 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:32.943517 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:32.943517 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:32.943517 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:32.943517 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:32.943517 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:32.943517 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 22:26:32.943517 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 22:26:32.943517 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 22:26:32.943517 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 22:26:32.944592 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:32.944592 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:32.944592 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 22:26:32.944592 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 22:26:32.944592 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 22:26:32.944592 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 22:26:32.944592 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:32.944592 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:32.944592 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:32.944592 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:32.944592 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:32.944592 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:32.944592 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:32.944592 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:32.944592 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:32.944592 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:32.944592 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:32.944592 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:32.945524 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:32.945524 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:32.945524 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 22:26:32.945524 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 22:26:32.945524 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 22:26:32.945524 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 22:26:32.945524 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 22:26:32.945524 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 22:26:32.945524 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 20 22:26:32.945524 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:26:32.945524 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 326/634] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Jul 20 22:26:32 2022] GET /identity/v3/users/glance => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:32.946118 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc260fa-f6be-48ae-bbec-7030311a047f None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:32.946118 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc260fa-f6be-48ae-bbec-7030311a047f None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:32.946456 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc260fa-f6be-48ae-bbec-7030311a047f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:32.962406 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-bdc260fa-f6be-48ae-bbec-7030311a047f None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:26:32.963613 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-bdc260fa-f6be-48ae-bbec-7030311a047f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:26:32.982804 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:32.998106 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f237a31-b750-4ee4-812b-058f05a894d5 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:33.030339 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 327/635] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:26:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 22:26:33.031193 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc260fa-f6be-48ae-bbec-7030311a047f None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:33.044624 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454d306d-a0c7-4528-8a6c-f99ed5b15f6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:33.045517 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454d306d-a0c7-4528-8a6c-f99ed5b15f6d None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:33.045874 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454d306d-a0c7-4528-8a6c-f99ed5b15f6d None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:33.046228 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454d306d-a0c7-4528-8a6c-f99ed5b15f6d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:33.056486 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454d306d-a0c7-4528-8a6c-f99ed5b15f6d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:33.057928 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454d306d-a0c7-4528-8a6c-f99ed5b15f6d None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:33.075510 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-454d306d-a0c7-4528-8a6c-f99ed5b15f6d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:26:33.075510 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:33.075510 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:33.075510 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:33.075510 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:33.075510 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:33.075510 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:33.075510 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:33.075510 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:33.075510 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:33.075510 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:33.075510 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:33.075510 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 22:26:33.075510 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 22:26:33.075510 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 22:26:33.076443 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 22:26:33.076443 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:33.076443 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:33.076443 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:33.076443 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:33.076443 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:33.076443 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:33.076443 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:33.076443 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:33.076443 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:33.076443 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:33.076443 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:33.076443 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:33.076443 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:33.076443 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:33.076443 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:33.076443 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:33.077319 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 22:26:33.077319 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 22:26:33.077319 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:26:33.077319 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:26:33.077319 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 309/636] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Jul 20 22:26:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:33.078045 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff15f62-2dea-4b7a-9c35-764481fb9f9b None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:33.078147 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff15f62-2dea-4b7a-9c35-764481fb9f9b None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:33.078430 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff15f62-2dea-4b7a-9c35-764481fb9f9b None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:33.083050 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 310/637] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:26:33 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:26:33.083050 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff15f62-2dea-4b7a-9c35-764481fb9f9b None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:33.134006 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc260fa-f6be-48ae-bbec-7030311a047f None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:33.143144 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:33.168203 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0469ba49-e4ac-4f2e-b68c-26832de3cabe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:33.169070 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0469ba49-e4ac-4f2e-b68c-26832de3cabe None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:33.169359 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0469ba49-e4ac-4f2e-b68c-26832de3cabe None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:33.169644 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0469ba49-e4ac-4f2e-b68c-26832de3cabe None admin] PATH_INFO: `/v3/projects/service` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:33.176662 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0469ba49-e4ac-4f2e-b68c-26832de3cabe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:33.178426 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0469ba49-e4ac-4f2e-b68c-26832de3cabe None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:33.179884 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff15f62-2dea-4b7a-9c35-764481fb9f9b None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:33.196618 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-0469ba49-e4ac-4f2e-b68c-26832de3cabe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 22:26:33.196618 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:33.196618 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:33.196618 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:33.196618 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:33.196618 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:33.196618 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:33.196618 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:33.196618 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:33.196618 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:33.196618 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:33.196618 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:33.196618 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 22:26:33.196618 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 22:26:33.196618 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 22:26:33.197594 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 22:26:33.197594 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:33.197594 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:33.197594 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:33.197594 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:33.197594 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:33.197594 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:33.197594 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:33.197594 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:33.197594 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:33.197594 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:33.197594 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:33.197594 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:33.197594 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:33.197594 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:33.197594 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:33.197594 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:33.198481 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 22:26:33.198481 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 22:26:33.198481 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 22:26:33.198481 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 22:26:33.198481 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 22:26:33.198481 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 22:26:33.198481 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 22:26:33.198481 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:26:33.198481 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 328/638] 10.222.0.81 () {56 vars in 1111 bytes} [Wed Jul 20 22:26:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:33.198983 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0469ba49-e4ac-4f2e-b68c-26832de3cabe None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:33.215202 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463ce518-55ed-4dff-a852-5c12a02be875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:33.215881 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463ce518-55ed-4dff-a852-5c12a02be875 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:33.216172 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463ce518-55ed-4dff-a852-5c12a02be875 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:33.216451 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463ce518-55ed-4dff-a852-5c12a02be875 None admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:33.217279 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463ce518-55ed-4dff-a852-5c12a02be875 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:33.217589 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463ce518-55ed-4dff-a852-5c12a02be875 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:33.219058 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463ce518-55ed-4dff-a852-5c12a02be875 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:33.221495 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:33.236002 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 311/639] 10.222.0.81 () {56 vars in 1137 bytes} [Wed Jul 20 22:26:33 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:33.236802 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463ce518-55ed-4dff-a852-5c12a02be875 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:33.300087 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0469ba49-e4ac-4f2e-b68c-26832de3cabe None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:33.333255 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167920e1-5fda-444e-b1eb-6a82b747ead3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:33.333946 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463ce518-55ed-4dff-a852-5c12a02be875 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:33.334414 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167920e1-5fda-444e-b1eb-6a82b747ead3 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:33.334414 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167920e1-5fda-444e-b1eb-6a82b747ead3 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:33.334633 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167920e1-5fda-444e-b1eb-6a82b747ead3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:33.344605 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167920e1-5fda-444e-b1eb-6a82b747ead3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 22:26:33.353522 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-167920e1-5fda-444e-b1eb-6a82b747ead3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 22:26:33.353522 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:33.353522 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:33.353522 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:33.353522 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:33.353522 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:33.353522 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:33.353522 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:33.353522 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:33.353522 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:33.353522 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:33.353522 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:33.353522 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 22:26:33.353522 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 22:26:33.353522 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 22:26:33.353522 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 22:26:33.354511 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 22:26:33.354511 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 22:26:33.354511 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 22:26:33.354511 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 22:26:33.354511 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:33.354511 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:33.354511 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:33.354511 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:33.354511 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:33.354511 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:33.354511 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:33.354511 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:33.354511 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:33.354511 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:33.354511 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:33.354511 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:33.355571 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:33.355571 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:33.355571 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:33.355571 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:33.355571 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 22:26:33.355571 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 22:26:33.355571 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 22:26:33.355571 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 22:26:33.355571 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 22:26:33.355571 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 22:26:33.355571 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 22:26:33.355571 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:26:33.355571 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 329/640] 10.222.0.81 () {56 vars in 1102 bytes} [Wed Jul 20 22:26:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:33.356521 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4571f3c5-d0db-42b1-ad62-eb1a4f567c12 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:33.356521 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4571f3c5-d0db-42b1-ad62-eb1a4f567c12 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:33.356521 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4571f3c5-d0db-42b1-ad62-eb1a4f567c12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:33.369297 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ce5391-aa80-4025-bc77-911824541a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:33.369973 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ce5391-aa80-4025-bc77-911824541a1f None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:33.370261 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ce5391-aa80-4025-bc77-911824541a1f None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:33.370540 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ce5391-aa80-4025-bc77-911824541a1f None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:33.371292 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ce5391-aa80-4025-bc77-911824541a1f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:33.371606 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ce5391-aa80-4025-bc77-911824541a1f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:33.372977 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ce5391-aa80-4025-bc77-911824541a1f None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:33.373724 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-4571f3c5-d0db-42b1-ad62-eb1a4f567c12 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:26:33.374913 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-4571f3c5-d0db-42b1-ad62-eb1a4f567c12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:26:33.384517 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 312/641] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Jul 20 22:26:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:33.385319 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ce5391-aa80-4025-bc77-911824541a1f None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:33.400147 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:33.440601 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:33.445375 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 330/642] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:26:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 22:26:33.446128 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4571f3c5-d0db-42b1-ad62-eb1a4f567c12 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:33.486151 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ce5391-aa80-4025-bc77-911824541a1f None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:33.519131 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431695e7-048d-4fb2-8e3b-d56f838291c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:33.520310 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431695e7-048d-4fb2-8e3b-d56f838291c6 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:33.520598 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431695e7-048d-4fb2-8e3b-d56f838291c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:33.520876 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431695e7-048d-4fb2-8e3b-d56f838291c6 None admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:33.521645 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431695e7-048d-4fb2-8e3b-d56f838291c6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:33.521950 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431695e7-048d-4fb2-8e3b-d56f838291c6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:33.523394 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431695e7-048d-4fb2-8e3b-d56f838291c6 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:33.534920 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 313/643] 10.222.0.81 () {56 vars in 1148 bytes} [Wed Jul 20 22:26:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:33.536432 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431695e7-048d-4fb2-8e3b-d56f838291c6 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:33.559365 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4571f3c5-d0db-42b1-ad62-eb1a4f567c12 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:33.593794 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c04de4-0394-4f4b-87a2-b3b2ffb81e8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:33.594443 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c04de4-0394-4f4b-87a2-b3b2ffb81e8c None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:33.594731 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c04de4-0394-4f4b-87a2-b3b2ffb81e8c None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:33.595027 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c04de4-0394-4f4b-87a2-b3b2ffb81e8c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:33.621329 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:33.632541 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431695e7-048d-4fb2-8e3b-d56f838291c6 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:33.647378 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:33.666037 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c04de4-0394-4f4b-87a2-b3b2ffb81e8c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:33.667585 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c04de4-0394-4f4b-87a2-b3b2ffb81e8c None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:33.669774 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8a9b92-838f-4ba4-824f-b066614d2dac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:33.670458 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8a9b92-838f-4ba4-824f-b066614d2dac None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:33.670746 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8a9b92-838f-4ba4-824f-b066614d2dac None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:33.671059 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8a9b92-838f-4ba4-824f-b066614d2dac None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:33.671792 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8a9b92-838f-4ba4-824f-b066614d2dac None admin] RBAC: Adding query filter params (name=service) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:33.672122 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8a9b92-838f-4ba4-824f-b066614d2dac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:33.673487 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8a9b92-838f-4ba4-824f-b066614d2dac None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:33.681739 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 314/644] 10.222.0.81 () {56 vars in 1103 bytes} [Wed Jul 20 22:26:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:33.682545 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8a9b92-838f-4ba4-824f-b066614d2dac None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:33.763688 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 331/645] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Jul 20 22:26:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 22:26:33.764553 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c04de4-0394-4f4b-87a2-b3b2ffb81e8c None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:33.790404 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8a9b92-838f-4ba4-824f-b066614d2dac None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:33.823068 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:33.824361 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3283488b-5ac2-4c77-9357-218c87ddddc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:33.825163 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3283488b-5ac2-4c77-9357-218c87ddddc1 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:33.825443 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3283488b-5ac2-4c77-9357-218c87ddddc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:33.825713 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3283488b-5ac2-4c77-9357-218c87ddddc1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:33.832779 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3283488b-5ac2-4c77-9357-218c87ddddc1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:33.834296 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3283488b-5ac2-4c77-9357-218c87ddddc1 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:33.849545 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-3283488b-5ac2-4c77-9357-218c87ddddc1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 22:26:33.849545 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:33.849545 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:33.849545 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:33.849545 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:33.849545 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:33.849545 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:33.849545 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:33.849545 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:33.849545 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:33.849545 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:33.849545 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:33.849545 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 22:26:33.849545 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 22:26:33.849545 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 22:26:33.850604 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 22:26:33.850604 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:33.850604 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:33.850604 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:33.850604 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:33.850604 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:33.850604 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:33.850604 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:33.850604 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:33.850604 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:33.850604 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:33.850604 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:33.850604 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:33.850604 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:33.850604 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:33.850604 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:33.850604 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:33.851733 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 22:26:33.851733 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 22:26:33.851733 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 22:26:33.851733 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 22:26:33.851733 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 22:26:33.851733 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 22:26:33.851733 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 22:26:33.851733 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:26:33.851733 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 315/646] 10.222.0.81 () {56 vars in 1111 bytes} [Wed Jul 20 22:26:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:33.851733 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3283488b-5ac2-4c77-9357-218c87ddddc1 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:33.867805 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c04de4-0394-4f4b-87a2-b3b2ffb81e8c None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:33.888836 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:33.902225 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c367d99-b296-4393-9eed-241e99987a05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:33.903077 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c367d99-b296-4393-9eed-241e99987a05 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:33.906531 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c367d99-b296-4393-9eed-241e99987a05 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:33.906531 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c367d99-b296-4393-9eed-241e99987a05 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:33.915031 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c367d99-b296-4393-9eed-241e99987a05 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 22:26:33.923701 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-0c367d99-b296-4393-9eed-241e99987a05 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 22:26:33.923701 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:33.923701 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:33.923701 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:33.923701 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:33.923701 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:33.923701 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:33.923701 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:33.923701 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:33.923701 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:33.923701 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:33.923701 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:33.923701 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 22:26:33.923701 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 22:26:33.923701 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 22:26:33.923701 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 22:26:33.924669 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 22:26:33.924669 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 22:26:33.924669 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 22:26:33.924669 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 22:26:33.924669 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:33.924669 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:33.924669 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:33.924669 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:33.924669 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:33.924669 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:33.924669 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:33.924669 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:33.924669 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:33.924669 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:33.924669 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:33.924669 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:33.925500 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:33.925500 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:33.925500 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:33.925500 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:33.925500 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 22:26:33.925500 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 22:26:33.925500 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 22:26:33.925500 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 22:26:33.925500 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 22:26:33.925500 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 22:26:33.925500 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 22:26:33.925500 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:26:33.925500 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 332/647] 10.222.0.81 () {56 vars in 1102 bytes} [Wed Jul 20 22:26:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:33.926194 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c367d99-b296-4393-9eed-241e99987a05 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:33.954723 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3283488b-5ac2-4c77-9357-218c87ddddc1 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:33.988309 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fdf332-f068-4773-8e50-c05754dc3c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:33.989213 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fdf332-f068-4773-8e50-c05754dc3c22 None admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:33.989506 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fdf332-f068-4773-8e50-c05754dc3c22 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:33.989785 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fdf332-f068-4773-8e50-c05754dc3c22 None admin] PATH_INFO: `/v3/projects/385edf2e59e546d090ace36600e29032/users/70e97436a08543a180225561ef8bc817/roles/9b4a83e2eca44a8da9f356bc40f8788a` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:34.003767 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fdf332-f068-4773-8e50-c05754dc3c22 None admin] RBAC: Authorizing `identity:create_grant(project_id=385edf2e59e546d090ace36600e29032, user_id=70e97436a08543a180225561ef8bc817, role_id=9b4a83e2eca44a8da9f356bc40f8788a)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:34.005717 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fdf332-f068-4773-8e50-c05754dc3c22 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:34.030688 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c367d99-b296-4393-9eed-241e99987a05 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:34.047124 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:34.062947 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c7c6d2-7205-4325-a770-1d3becada584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:34.063776 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c7c6d2-7205-4325-a770-1d3becada584 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:34.064061 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c7c6d2-7205-4325-a770-1d3becada584 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:34.064343 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c7c6d2-7205-4325-a770-1d3becada584 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:34.074977 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c7c6d2-7205-4325-a770-1d3becada584 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:34.076450 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c7c6d2-7205-4325-a770-1d3becada584 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:34.084656 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 316/648] 10.222.0.81 () {58 vars in 1439 bytes} [Wed Jul 20 22:26:33 2022] PUT /identity/v3/projects/385edf2e59e546d090ace36600e29032/users/70e97436a08543a180225561ef8bc817/roles/9b4a83e2eca44a8da9f356bc40f8788a => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:26:34.085308 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fdf332-f068-4773-8e50-c05754dc3c22 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:34.095708 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-32c7c6d2-7205-4325-a770-1d3becada584 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:26:34.095708 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:34.095708 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:34.095708 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:34.095708 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:34.095708 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:34.095708 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:34.095708 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:34.095708 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:34.095708 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:34.095708 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:34.095708 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:34.095708 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 22:26:34.095708 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 22:26:34.095708 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 22:26:34.096946 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 22:26:34.096946 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:34.096946 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:34.096946 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:34.096946 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:34.096946 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:34.096946 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:34.096946 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:34.096946 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:34.096946 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:34.096946 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:34.096946 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:34.096946 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:34.096946 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:34.096946 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:34.096946 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:34.096946 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:34.097871 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 22:26:34.097871 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 22:26:34.097871 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:26:34.097871 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:26:34.106249 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 333/649] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Jul 20 22:26:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:34.106249 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c7c6d2-7205-4325-a770-1d3becada584 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:34.201073 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:34.217072 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fdf332-f068-4773-8e50-c05754dc3c22 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:34.223070 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c7c6d2-7205-4325-a770-1d3becada584 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:34.258341 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8174557-b1bd-414a-8dfb-81bf6cf57061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:34.259207 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8174557-b1bd-414a-8dfb-81bf6cf57061 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:34.259495 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8174557-b1bd-414a-8dfb-81bf6cf57061 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:34.259775 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8174557-b1bd-414a-8dfb-81bf6cf57061 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:34.265805 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439d2610-ecfe-4e8d-bf40-c6807e1216d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:34.266667 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439d2610-ecfe-4e8d-bf40-c6807e1216d3 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:34.266955 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439d2610-ecfe-4e8d-bf40-c6807e1216d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:34.267270 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439d2610-ecfe-4e8d-bf40-c6807e1216d3 None admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:34.268061 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439d2610-ecfe-4e8d-bf40-c6807e1216d3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:34.268375 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439d2610-ecfe-4e8d-bf40-c6807e1216d3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:34.269853 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439d2610-ecfe-4e8d-bf40-c6807e1216d3 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:34.270339 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8174557-b1bd-414a-8dfb-81bf6cf57061 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:34.271822 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8174557-b1bd-414a-8dfb-81bf6cf57061 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:34.282884 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 334/650] 10.222.0.81 () {56 vars in 1148 bytes} [Wed Jul 20 22:26:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:34.283601 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439d2610-ecfe-4e8d-bf40-c6807e1216d3 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:34.297489 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-f8174557-b1bd-414a-8dfb-81bf6cf57061 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:26:34.297489 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:34.297489 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:34.297489 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:34.297489 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:34.297489 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:34.297489 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:34.297489 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:34.297489 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:34.297489 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:34.297489 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:34.297489 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:34.297489 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 22:26:34.297489 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 22:26:34.297489 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 22:26:34.298419 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 22:26:34.298419 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:34.298419 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:34.298419 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:34.298419 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:34.298419 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:34.298419 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:34.298419 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:34.298419 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:34.298419 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:34.298419 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:34.298419 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:34.298419 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:34.298419 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:34.298419 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:34.298419 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:34.298419 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:34.299330 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 22:26:34.299330 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 22:26:34.299330 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:26:34.299330 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:26:34.303081 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 317/651] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Jul 20 22:26:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:34.304280 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8174557-b1bd-414a-8dfb-81bf6cf57061 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:34.317399 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:34.391083 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439d2610-ecfe-4e8d-bf40-c6807e1216d3 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:34.408116 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8174557-b1bd-414a-8dfb-81bf6cf57061 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:34.428135 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ab6e65-f101-40ed-a273-7d67aa95c3af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:34.428814 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ab6e65-f101-40ed-a273-7d67aa95c3af None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:34.429100 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ab6e65-f101-40ed-a273-7d67aa95c3af None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:34.429794 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ab6e65-f101-40ed-a273-7d67aa95c3af None admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:34.430551 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ab6e65-f101-40ed-a273-7d67aa95c3af None admin] RBAC: Adding query filter params (name=service) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:34.431032 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ab6e65-f101-40ed-a273-7d67aa95c3af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:34.432413 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ab6e65-f101-40ed-a273-7d67aa95c3af None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:34.440867 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 335/652] 10.222.0.81 () {56 vars in 1103 bytes} [Wed Jul 20 22:26:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:34.442360 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ab6e65-f101-40ed-a273-7d67aa95c3af None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:34.448934 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d7c961-07cd-44b0-9ac6-3420b3448422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:34.449615 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d7c961-07cd-44b0-9ac6-3420b3448422 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:34.449901 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d7c961-07cd-44b0-9ac6-3420b3448422 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:34.450180 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d7c961-07cd-44b0-9ac6-3420b3448422 None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:34.450904 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d7c961-07cd-44b0-9ac6-3420b3448422 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:34.451245 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d7c961-07cd-44b0-9ac6-3420b3448422 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:34.452624 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d7c961-07cd-44b0-9ac6-3420b3448422 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:34.459522 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:34.469255 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 318/653] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Jul 20 22:26:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:34.469255 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d7c961-07cd-44b0-9ac6-3420b3448422 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:34.541899 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:34.552870 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ab6e65-f101-40ed-a273-7d67aa95c3af None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:34.575114 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d7c961-07cd-44b0-9ac6-3420b3448422 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:34.591150 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a1a140-f048-44b0-85d6-98f6e49e803b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:34.592406 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a1a140-f048-44b0-85d6-98f6e49e803b None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:34.592870 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a1a140-f048-44b0-85d6-98f6e49e803b None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:34.593343 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a1a140-f048-44b0-85d6-98f6e49e803b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:34.594354 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a1a140-f048-44b0-85d6-98f6e49e803b None admin] RBAC: Adding query filter params (role.id=9b4a83e2eca44a8da9f356bc40f8788a, scope.project.id=385edf2e59e546d090ace36600e29032, user.id=7829324db2574d38b18c749d0559177f) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:34.594840 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a1a140-f048-44b0-85d6-98f6e49e803b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:34.596464 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a1a140-f048-44b0-85d6-98f6e49e803b None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:34.602965 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 336/654] 10.222.0.81 () {56 vars in 1374 bytes} [Wed Jul 20 22:26:34 2022] GET /identity/v3/role_assignments?user.id=7829324db2574d38b18c749d0559177f&scope.project.id=385edf2e59e546d090ace36600e29032&role.id=9b4a83e2eca44a8da9f356bc40f8788a => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:34.603847 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a1a140-f048-44b0-85d6-98f6e49e803b None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:34.624915 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ea9920-255f-4b08-93fd-7fd7a5f176f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:34.625944 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ea9920-255f-4b08-93fd-7fd7a5f176f0 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:34.626448 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ea9920-255f-4b08-93fd-7fd7a5f176f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:34.626906 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ea9920-255f-4b08-93fd-7fd7a5f176f0 None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:34.628625 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ea9920-255f-4b08-93fd-7fd7a5f176f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:34.630008 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ea9920-255f-4b08-93fd-7fd7a5f176f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:34.632563 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ea9920-255f-4b08-93fd-7fd7a5f176f0 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:34.654580 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 319/655] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Jul 20 22:26:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:34.655515 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ea9920-255f-4b08-93fd-7fd7a5f176f0 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:34.723805 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a1a140-f048-44b0-85d6-98f6e49e803b None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:34.743459 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:34.762385 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2191ab-055c-4d60-b007-6b4a346bebbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:34.763257 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2191ab-055c-4d60-b007-6b4a346bebbd None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:34.763542 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2191ab-055c-4d60-b007-6b4a346bebbd None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:34.763820 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2191ab-055c-4d60-b007-6b4a346bebbd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:34.772533 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ea9920-255f-4b08-93fd-7fd7a5f176f0 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:34.776032 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2191ab-055c-4d60-b007-6b4a346bebbd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:34.777515 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2191ab-055c-4d60-b007-6b4a346bebbd None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:34.798226 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-2a2191ab-055c-4d60-b007-6b4a346bebbd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:26:34.798226 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:34.798226 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:34.798226 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:34.798226 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:34.798226 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:34.798226 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:34.798226 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:34.798226 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:34.798226 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:34.798226 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:34.798226 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:34.798226 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 22:26:34.798226 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 22:26:34.798226 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 22:26:34.799188 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 22:26:34.799188 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:34.799188 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:34.799188 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:34.799188 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:34.799188 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:34.799188 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:34.799188 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:34.799188 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:34.799188 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:34.799188 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:34.799188 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:34.799188 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:34.799188 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:34.799188 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:34.799188 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:34.799188 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:34.800076 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 22:26:34.800076 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 22:26:34.800076 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:26:34.800076 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:26:34.800076 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 337/656] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Jul 20 22:26:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:34.800789 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2191ab-055c-4d60-b007-6b4a346bebbd None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:34.814615 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:34.819506 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091d5390-0c31-4d1f-9ce8-b1e2deed7fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:34.820343 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091d5390-0c31-4d1f-9ce8-b1e2deed7fba None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:34.820633 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091d5390-0c31-4d1f-9ce8-b1e2deed7fba None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:34.820914 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091d5390-0c31-4d1f-9ce8-b1e2deed7fba None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:34.829280 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091d5390-0c31-4d1f-9ce8-b1e2deed7fba None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:34.830876 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091d5390-0c31-4d1f-9ce8-b1e2deed7fba None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:34.848932 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-091d5390-0c31-4d1f-9ce8-b1e2deed7fba None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 20 22:26:34.848932 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:34.848932 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:34.848932 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:34.848932 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:34.848932 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:34.848932 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:34.848932 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:34.848932 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:34.848932 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:34.848932 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:34.848932 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:34.848932 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 22:26:34.848932 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 22:26:34.848932 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 22:26:34.848932 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 22:26:34.849912 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:34.849912 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:34.849912 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 22:26:34.849912 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 22:26:34.849912 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 22:26:34.849912 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 22:26:34.849912 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:34.849912 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:34.849912 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:34.849912 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:34.849912 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:34.849912 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:34.849912 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:34.849912 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:34.849912 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:34.849912 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:34.849912 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:34.849912 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:34.850866 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:34.850866 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:34.850866 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 22:26:34.850866 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 22:26:34.850866 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 22:26:34.850866 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 22:26:34.850866 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 22:26:34.850866 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 22:26:34.850866 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 20 22:26:34.850866 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:26:34.850866 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 320/657] 10.222.0.81 () {56 vars in 1102 bytes} [Wed Jul 20 22:26:34 2022] GET /identity/v3/users/neutron => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:34.853809 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091d5390-0c31-4d1f-9ce8-b1e2deed7fba None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:34.921724 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2191ab-055c-4d60-b007-6b4a346bebbd None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:34.966881 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0c258e-2d96-41bd-a000-5742e7912a07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:34.967832 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0c258e-2d96-41bd-a000-5742e7912a07 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:34.968182 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0c258e-2d96-41bd-a000-5742e7912a07 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:34.968529 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0c258e-2d96-41bd-a000-5742e7912a07 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:34.976532 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0c258e-2d96-41bd-a000-5742e7912a07 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:34.978058 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0c258e-2d96-41bd-a000-5742e7912a07 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:34.985755 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091d5390-0c31-4d1f-9ce8-b1e2deed7fba None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:34.992741 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-4d0c258e-2d96-41bd-a000-5742e7912a07 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 22:26:34.992741 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:34.992741 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:34.992741 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:34.992741 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:34.992741 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:34.992741 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:34.992741 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:34.992741 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:34.992741 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:34.992741 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:34.992741 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:34.992741 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 22:26:34.992741 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 22:26:34.992741 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 22:26:34.993627 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 22:26:34.993627 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:34.993627 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:34.993627 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:34.993627 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:34.993627 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:34.993627 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:34.993627 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:34.993627 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:34.993627 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:34.993627 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:34.993627 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:34.993627 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:34.993627 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:34.993627 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:34.993627 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:34.993627 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:34.994498 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 22:26:34.994498 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 22:26:34.994498 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 22:26:34.994498 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 22:26:34.994498 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 22:26:34.994498 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 22:26:34.994498 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 22:26:34.994498 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:26:34.994498 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 338/658] 10.222.0.81 () {56 vars in 1111 bytes} [Wed Jul 20 22:26:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:34.994985 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0c258e-2d96-41bd-a000-5742e7912a07 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:35.011967 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:35.035903 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff62109-85cc-40d4-a859-bbb09091b333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:35.037460 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff62109-85cc-40d4-a859-bbb09091b333 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:35.037747 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff62109-85cc-40d4-a859-bbb09091b333 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:35.038025 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff62109-85cc-40d4-a859-bbb09091b333 None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:35.038747 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff62109-85cc-40d4-a859-bbb09091b333 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:35.039076 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff62109-85cc-40d4-a859-bbb09091b333 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:35.040703 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff62109-85cc-40d4-a859-bbb09091b333 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:35.057295 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 321/659] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Jul 20 22:26:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:35.058496 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff62109-85cc-40d4-a859-bbb09091b333 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:35.075303 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:35.114174 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0c258e-2d96-41bd-a000-5742e7912a07 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:35.151776 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de9a968-a180-4c4d-92a9-e63185015fc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:35.152462 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de9a968-a180-4c4d-92a9-e63185015fc3 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:35.152746 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de9a968-a180-4c4d-92a9-e63185015fc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:35.153025 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de9a968-a180-4c4d-92a9-e63185015fc3 None admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:35.153926 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de9a968-a180-4c4d-92a9-e63185015fc3 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:35.154242 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de9a968-a180-4c4d-92a9-e63185015fc3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:35.155701 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de9a968-a180-4c4d-92a9-e63185015fc3 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:35.177733 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 339/660] 10.222.0.81 () {56 vars in 1139 bytes} [Wed Jul 20 22:26:34 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:35.183653 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de9a968-a180-4c4d-92a9-e63185015fc3 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:35.195656 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff62109-85cc-40d4-a859-bbb09091b333 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:35.243872 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f939f0a9-0b34-46a9-a74f-9d1156a453b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:35.244557 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f939f0a9-0b34-46a9-a74f-9d1156a453b4 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:35.244849 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f939f0a9-0b34-46a9-a74f-9d1156a453b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:35.245128 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f939f0a9-0b34-46a9-a74f-9d1156a453b4 None admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:35.245925 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f939f0a9-0b34-46a9-a74f-9d1156a453b4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:35.246236 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f939f0a9-0b34-46a9-a74f-9d1156a453b4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:35.247727 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f939f0a9-0b34-46a9-a74f-9d1156a453b4 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:35.261683 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 322/661] 10.222.0.81 () {56 vars in 1148 bytes} [Wed Jul 20 22:26:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:35.262697 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f939f0a9-0b34-46a9-a74f-9d1156a453b4 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:35.314571 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:35.327666 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de9a968-a180-4c4d-92a9-e63185015fc3 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:35.354178 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:35.362931 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff55e94e-ad99-40df-99bd-684a6a388716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:35.364307 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff55e94e-ad99-40df-99bd-684a6a388716 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:35.364597 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff55e94e-ad99-40df-99bd-684a6a388716 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:35.372993 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff55e94e-ad99-40df-99bd-684a6a388716 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:35.372993 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff55e94e-ad99-40df-99bd-684a6a388716 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:35.373909 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff55e94e-ad99-40df-99bd-684a6a388716 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:35.383734 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f939f0a9-0b34-46a9-a74f-9d1156a453b4 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:35.389499 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-ff55e94e-ad99-40df-99bd-684a6a388716 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 20 22:26:35.389499 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:35.389499 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:35.389499 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:35.389499 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:35.389499 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:35.389499 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:35.389499 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:35.389499 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:35.389499 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:35.389499 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:35.389499 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:35.389499 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 22:26:35.389499 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 22:26:35.389499 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 22:26:35.389499 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 22:26:35.390517 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:35.390517 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:35.390517 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 22:26:35.390517 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 22:26:35.390517 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 22:26:35.390517 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 22:26:35.390517 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:35.390517 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:35.390517 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:35.390517 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:35.390517 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:35.390517 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:35.390517 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:35.390517 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:35.390517 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:35.390517 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:35.390517 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:35.390517 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:35.391516 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:35.391516 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:35.391516 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 22:26:35.391516 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 22:26:35.391516 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 22:26:35.391516 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 22:26:35.391516 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 22:26:35.391516 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 22:26:35.391516 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 20 22:26:35.391516 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:26:35.395318 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 340/662] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Jul 20 22:26:35 2022] GET /identity/v3/users/cinder => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:35.396167 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff55e94e-ad99-40df-99bd-684a6a388716 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:35.425090 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2cd4e8-cf27-4ad0-b8e5-4afe82cf706b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:35.427073 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2cd4e8-cf27-4ad0-b8e5-4afe82cf706b None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:35.427567 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2cd4e8-cf27-4ad0-b8e5-4afe82cf706b None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:35.428066 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2cd4e8-cf27-4ad0-b8e5-4afe82cf706b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:35.501668 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2cd4e8-cf27-4ad0-b8e5-4afe82cf706b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:35.502648 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff55e94e-ad99-40df-99bd-684a6a388716 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:35.503849 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2cd4e8-cf27-4ad0-b8e5-4afe82cf706b None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:35.538771 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455a2de5-9fe4-4a85-af54-86e7f25af75c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:35.539792 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455a2de5-9fe4-4a85-af54-86e7f25af75c None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:35.540086 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455a2de5-9fe4-4a85-af54-86e7f25af75c None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:35.540366 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455a2de5-9fe4-4a85-af54-86e7f25af75c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:35.541197 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455a2de5-9fe4-4a85-af54-86e7f25af75c None admin] RBAC: Adding query filter params (role.id=9b4a83e2eca44a8da9f356bc40f8788a, scope.project.id=385edf2e59e546d090ace36600e29032, user.id=9c44640ea0014426a9523bb1c457778b) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:35.541512 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455a2de5-9fe4-4a85-af54-86e7f25af75c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:35.543440 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455a2de5-9fe4-4a85-af54-86e7f25af75c None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:35.548073 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:35.549447 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 341/663] 10.222.0.81 () {56 vars in 1374 bytes} [Wed Jul 20 22:26:35 2022] GET /identity/v3/role_assignments?user.id=9c44640ea0014426a9523bb1c457778b&scope.project.id=385edf2e59e546d090ace36600e29032&role.id=9b4a83e2eca44a8da9f356bc40f8788a => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:35.550270 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455a2de5-9fe4-4a85-af54-86e7f25af75c None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:35.609445 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:35.640476 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 323/664] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Jul 20 22:26:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 22:26:35.648331 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2cd4e8-cf27-4ad0-b8e5-4afe82cf706b None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:35.689119 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455a2de5-9fe4-4a85-af54-86e7f25af75c None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:35.771140 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272ec007-4631-4f37-80a2-bdecb22eb402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:35.771841 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272ec007-4631-4f37-80a2-bdecb22eb402 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:35.772115 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272ec007-4631-4f37-80a2-bdecb22eb402 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:35.772396 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272ec007-4631-4f37-80a2-bdecb22eb402 None admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:35.773354 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272ec007-4631-4f37-80a2-bdecb22eb402 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:35.773670 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272ec007-4631-4f37-80a2-bdecb22eb402 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:35.775772 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272ec007-4631-4f37-80a2-bdecb22eb402 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:35.798539 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 342/665] 10.222.0.81 () {56 vars in 1137 bytes} [Wed Jul 20 22:26:35 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:35.803545 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272ec007-4631-4f37-80a2-bdecb22eb402 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:35.812258 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2cd4e8-cf27-4ad0-b8e5-4afe82cf706b None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:35.848165 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806591f2-21b4-40b3-b0d6-7afbf333dd3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:35.849336 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806591f2-21b4-40b3-b0d6-7afbf333dd3a None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:35.849823 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806591f2-21b4-40b3-b0d6-7afbf333dd3a None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:35.850324 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806591f2-21b4-40b3-b0d6-7afbf333dd3a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:35.858170 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:35.862212 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806591f2-21b4-40b3-b0d6-7afbf333dd3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:35.864443 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806591f2-21b4-40b3-b0d6-7afbf333dd3a None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:35.888306 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-806591f2-21b4-40b3-b0d6-7afbf333dd3a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:26:35.888306 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:35.888306 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:35.888306 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:35.888306 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:35.888306 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:35.888306 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:35.888306 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:35.888306 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:35.888306 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:35.888306 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:35.888306 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:35.888306 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 22:26:35.888306 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 22:26:35.888306 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 22:26:35.889451 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 22:26:35.889451 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:35.889451 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:35.889451 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:35.889451 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:35.889451 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:35.889451 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:35.889451 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:35.889451 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:35.889451 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:35.889451 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:35.889451 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:35.889451 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:35.889451 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:35.889451 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:35.889451 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:35.889451 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:35.890555 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 22:26:35.890555 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 22:26:35.890555 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:26:35.890555 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:26:35.891292 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 324/666] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Jul 20 22:26:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:35.896389 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806591f2-21b4-40b3-b0d6-7afbf333dd3a None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:35.922978 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272ec007-4631-4f37-80a2-bdecb22eb402 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:35.933872 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:35.960156 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e6302e-1784-457b-adf0-6b3116ec6a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:35.961424 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e6302e-1784-457b-adf0-6b3116ec6a8b None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:35.961424 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e6302e-1784-457b-adf0-6b3116ec6a8b None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:35.961623 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e6302e-1784-457b-adf0-6b3116ec6a8b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:36.020376 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806591f2-21b4-40b3-b0d6-7afbf333dd3a None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:36.048422 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e6302e-1784-457b-adf0-6b3116ec6a8b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:36.049931 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e6302e-1784-457b-adf0-6b3116ec6a8b None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:36.062569 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef044a0-38f5-4b07-9c20-a7520abbc7f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:36.063595 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef044a0-38f5-4b07-9c20-a7520abbc7f5 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:36.064109 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef044a0-38f5-4b07-9c20-a7520abbc7f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:36.064648 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef044a0-38f5-4b07-9c20-a7520abbc7f5 None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:36.065717 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef044a0-38f5-4b07-9c20-a7520abbc7f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:36.066225 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef044a0-38f5-4b07-9c20-a7520abbc7f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:36.067929 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef044a0-38f5-4b07-9c20-a7520abbc7f5 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:36.080915 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 325/667] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Jul 20 22:26:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:36.087972 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef044a0-38f5-4b07-9c20-a7520abbc7f5 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:36.140349 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:36.165271 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 343/668] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Jul 20 22:26:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 20 22:26:36.171886 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e6302e-1784-457b-adf0-6b3116ec6a8b None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:36.183733 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:36.209754 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef044a0-38f5-4b07-9c20-a7520abbc7f5 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:36.244319 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05994792-1577-4939-8240-df964296b9ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:36.245441 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05994792-1577-4939-8240-df964296b9ff None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:36.245908 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05994792-1577-4939-8240-df964296b9ff None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:36.246365 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05994792-1577-4939-8240-df964296b9ff None admin] PATH_INFO: `/v3/projects/service` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:36.253573 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05994792-1577-4939-8240-df964296b9ff None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:36.255813 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05994792-1577-4939-8240-df964296b9ff None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:36.271473 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-05994792-1577-4939-8240-df964296b9ff None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 22:26:36.271473 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:36.271473 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:36.271473 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:36.271473 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:36.271473 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:36.271473 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:36.271473 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:36.271473 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:36.271473 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:36.271473 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:36.271473 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:36.271473 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 22:26:36.271473 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 22:26:36.271473 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 22:26:36.272377 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 22:26:36.272377 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:36.272377 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:36.272377 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:36.272377 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:36.272377 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:36.272377 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:36.272377 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:36.272377 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:36.272377 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:36.272377 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:36.272377 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:36.272377 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:36.272377 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:36.272377 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:36.272377 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:36.272377 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:36.273243 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 22:26:36.273243 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 22:26:36.273243 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 22:26:36.273243 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 22:26:36.273243 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 22:26:36.273243 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 22:26:36.273243 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 22:26:36.273243 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:26:36.275696 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 326/669] 10.222.0.81 () {56 vars in 1111 bytes} [Wed Jul 20 22:26:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:36.283144 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05994792-1577-4939-8240-df964296b9ff None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:36.291588 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e6302e-1784-457b-adf0-6b3116ec6a8b None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:36.334975 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7582e4-321a-4c17-95b3-aab89401e493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:36.336087 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7582e4-321a-4c17-95b3-aab89401e493 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:36.336523 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7582e4-321a-4c17-95b3-aab89401e493 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:36.336953 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7582e4-321a-4c17-95b3-aab89401e493 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:36.348550 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7582e4-321a-4c17-95b3-aab89401e493 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:36.350375 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7582e4-321a-4c17-95b3-aab89401e493 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:36.370385 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-ae7582e4-321a-4c17-95b3-aab89401e493 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:26:36.370385 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:36.370385 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:36.370385 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:36.370385 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:36.370385 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:36.370385 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:36.370385 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:36.370385 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:36.370385 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:36.370385 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:36.370385 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:36.370385 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 22:26:36.370385 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 22:26:36.370385 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 22:26:36.371354 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 22:26:36.371354 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:36.371354 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:36.371354 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:36.371354 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:36.371354 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:36.371354 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:36.371354 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:36.371354 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:36.371354 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:36.371354 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:36.371354 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:36.371354 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:36.371354 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:36.371354 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:36.371354 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:36.371354 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:36.372251 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 22:26:36.372251 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 22:26:36.372251 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:26:36.372251 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:26:36.374434 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 344/670] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Jul 20 22:26:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 22:26:36.381187 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611c73f3-07fc-48f5-a2f2-665fe163fd0d None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:36.381801 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611c73f3-07fc-48f5-a2f2-665fe163fd0d None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:36.382265 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611c73f3-07fc-48f5-a2f2-665fe163fd0d None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:36.384431 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 345/671] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:26:36 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:26:36.385518 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611c73f3-07fc-48f5-a2f2-665fe163fd0d None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:36.412488 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05994792-1577-4939-8240-df964296b9ff None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:36.427446 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:36.451900 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c4f7fa-4ac8-4271-840d-b6e56033f7a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:36.452676 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c4f7fa-4ac8-4271-840d-b6e56033f7a3 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:36.453049 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c4f7fa-4ac8-4271-840d-b6e56033f7a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:36.453741 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c4f7fa-4ac8-4271-840d-b6e56033f7a3 None admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:36.454341 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c4f7fa-4ac8-4271-840d-b6e56033f7a3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:36.454723 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c4f7fa-4ac8-4271-840d-b6e56033f7a3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:36.456267 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c4f7fa-4ac8-4271-840d-b6e56033f7a3 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:36.468811 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:36.470373 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 327/672] 10.222.0.81 () {56 vars in 1148 bytes} [Wed Jul 20 22:26:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:36.471913 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184a9f89-682f-4d8d-85d5-caf7a5008c97 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:36.472389 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184a9f89-682f-4d8d-85d5-caf7a5008c97 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:36.473366 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184a9f89-682f-4d8d-85d5-caf7a5008c97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:36.490487 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-184a9f89-682f-4d8d-85d5-caf7a5008c97 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:26:36.492082 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-184a9f89-682f-4d8d-85d5-caf7a5008c97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:26:36.518524 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611c73f3-07fc-48f5-a2f2-665fe163fd0d None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:36.559381 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c437b503-2eca-4475-806e-c9261559b085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:36.560055 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c437b503-2eca-4475-806e-c9261559b085 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:36.560341 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c437b503-2eca-4475-806e-c9261559b085 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:36.560668 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c437b503-2eca-4475-806e-c9261559b085 None admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:36.561871 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c437b503-2eca-4475-806e-c9261559b085 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:36.562183 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c437b503-2eca-4475-806e-c9261559b085 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:36.563606 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c437b503-2eca-4475-806e-c9261559b085 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:36.569260 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 328/673] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:26:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 22:26:36.570407 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184a9f89-682f-4d8d-85d5-caf7a5008c97 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:36.576401 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 346/674] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Jul 20 22:26:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:36.581625 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c437b503-2eca-4475-806e-c9261559b085 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:36.690163 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:36.695242 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184a9f89-682f-4d8d-85d5-caf7a5008c97 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:36.698338 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c437b503-2eca-4475-806e-c9261559b085 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:36.719821 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:36.730897 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed353638-07c1-4b7f-b556-ffea2b33491a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:36.731722 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed353638-07c1-4b7f-b556-ffea2b33491a None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:36.732079 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed353638-07c1-4b7f-b556-ffea2b33491a None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:36.732637 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed353638-07c1-4b7f-b556-ffea2b33491a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:36.733388 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed353638-07c1-4b7f-b556-ffea2b33491a None admin] RBAC: Adding query filter params (role.id=9b4a83e2eca44a8da9f356bc40f8788a, scope.project.id=385edf2e59e546d090ace36600e29032, user.id=333c2b1b11cc4c478abfd965a72e9d66) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:36.733753 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed353638-07c1-4b7f-b556-ffea2b33491a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:36.735307 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed353638-07c1-4b7f-b556-ffea2b33491a None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:36.737970 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0802cbdf-4825-4006-8a22-e9c4298d395a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:36.739615 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0802cbdf-4825-4006-8a22-e9c4298d395a None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:36.740074 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0802cbdf-4825-4006-8a22-e9c4298d395a None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:36.740555 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0802cbdf-4825-4006-8a22-e9c4298d395a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:36.742956 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 329/675] 10.222.0.81 () {56 vars in 1374 bytes} [Wed Jul 20 22:26:36 2022] GET /identity/v3/role_assignments?user.id=333c2b1b11cc4c478abfd965a72e9d66&scope.project.id=385edf2e59e546d090ace36600e29032&role.id=9b4a83e2eca44a8da9f356bc40f8788a => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:36.743976 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed353638-07c1-4b7f-b556-ffea2b33491a None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:36.748965 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0802cbdf-4825-4006-8a22-e9c4298d395a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:36.753638 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0802cbdf-4825-4006-8a22-e9c4298d395a None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:36.769090 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-0802cbdf-4825-4006-8a22-e9c4298d395a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 22:26:36.769090 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:36.769090 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:36.769090 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:36.769090 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:36.769090 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:36.769090 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:36.769090 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:36.769090 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:36.769090 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:36.769090 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:36.769090 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:36.769090 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 22:26:36.769090 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 22:26:36.769090 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 22:26:36.770091 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 22:26:36.770091 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:36.770091 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:36.770091 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:36.770091 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:36.770091 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:36.770091 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:36.770091 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:36.770091 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:36.770091 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:36.770091 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:36.770091 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:36.770091 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:36.770091 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:36.770091 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:36.770091 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:36.770091 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:36.770988 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 22:26:36.770988 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 22:26:36.770988 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 22:26:36.770988 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 22:26:36.770988 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 22:26:36.770988 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 22:26:36.770988 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 22:26:36.770988 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:26:36.773370 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 347/676] 10.222.0.81 () {56 vars in 1111 bytes} [Wed Jul 20 22:26:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 22:26:36.779798 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0802cbdf-4825-4006-8a22-e9c4298d395a None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:36.856067 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed353638-07c1-4b7f-b556-ffea2b33491a None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:36.893457 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ffe443-6fb6-4f09-9ad4-390c1b07bec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:36.894613 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ffe443-6fb6-4f09-9ad4-390c1b07bec9 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:36.894728 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ffe443-6fb6-4f09-9ad4-390c1b07bec9 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:36.895090 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ffe443-6fb6-4f09-9ad4-390c1b07bec9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:36.904998 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0802cbdf-4825-4006-8a22-e9c4298d395a None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:36.906635 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ffe443-6fb6-4f09-9ad4-390c1b07bec9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 22:26:36.916338 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-f7ffe443-6fb6-4f09-9ad4-390c1b07bec9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 22:26:36.916338 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:36.916338 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:36.916338 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:36.916338 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:36.916338 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:36.916338 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:36.916338 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:36.916338 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:36.916338 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:36.916338 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:36.916338 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:36.916338 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 22:26:36.916338 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 22:26:36.916338 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 22:26:36.916338 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 22:26:36.917329 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 22:26:36.917329 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 22:26:36.917329 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 22:26:36.917329 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 22:26:36.917329 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:36.917329 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:36.917329 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:36.917329 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:36.917329 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:36.917329 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:36.917329 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:36.917329 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:36.917329 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:36.917329 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:36.917329 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:36.917329 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:36.918183 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:36.918183 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:36.918183 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:36.918183 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:36.918183 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 22:26:36.918183 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 22:26:36.918183 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 22:26:36.918183 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 22:26:36.918183 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 22:26:36.918183 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 22:26:36.918183 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 22:26:36.918183 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:26:36.918183 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 330/677] 10.222.0.81 () {56 vars in 1102 bytes} [Wed Jul 20 22:26:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:36.924130 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ffe443-6fb6-4f09-9ad4-390c1b07bec9 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:36.951873 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:36.962928 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429544d5-a73c-4b5b-ab9a-1998c4df336e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:36.964023 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429544d5-a73c-4b5b-ab9a-1998c4df336e None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:36.964486 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429544d5-a73c-4b5b-ab9a-1998c4df336e None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:36.971170 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429544d5-a73c-4b5b-ab9a-1998c4df336e None admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:36.971170 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429544d5-a73c-4b5b-ab9a-1998c4df336e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:36.971170 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429544d5-a73c-4b5b-ab9a-1998c4df336e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:36.971170 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429544d5-a73c-4b5b-ab9a-1998c4df336e None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:36.996058 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 348/678] 10.222.0.81 () {56 vars in 1148 bytes} [Wed Jul 20 22:26:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:37.004031 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429544d5-a73c-4b5b-ab9a-1998c4df336e None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:37.045761 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ffe443-6fb6-4f09-9ad4-390c1b07bec9 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:37.070316 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:37.096738 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da022219-761f-4cdb-9357-373632b34956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:37.097486 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da022219-761f-4cdb-9357-373632b34956 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:37.098474 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da022219-761f-4cdb-9357-373632b34956 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:37.098755 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da022219-761f-4cdb-9357-373632b34956 None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:37.099540 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da022219-761f-4cdb-9357-373632b34956 None admin] RBAC: Adding query filter params (name=service) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:37.099853 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da022219-761f-4cdb-9357-373632b34956 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:37.101245 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da022219-761f-4cdb-9357-373632b34956 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:37.110668 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 331/679] 10.222.0.81 () {56 vars in 1103 bytes} [Wed Jul 20 22:26:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:37.117689 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da022219-761f-4cdb-9357-373632b34956 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:37.142901 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429544d5-a73c-4b5b-ab9a-1998c4df336e None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:37.179462 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5f3cb2-a5fb-43b7-8600-4d810b86bdf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:37.180485 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5f3cb2-a5fb-43b7-8600-4d810b86bdf3 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:37.181001 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5f3cb2-a5fb-43b7-8600-4d810b86bdf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:37.181485 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5f3cb2-a5fb-43b7-8600-4d810b86bdf3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:37.182523 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5f3cb2-a5fb-43b7-8600-4d810b86bdf3 None admin] RBAC: Adding query filter params (role.id=9b4a83e2eca44a8da9f356bc40f8788a, scope.project.id=385edf2e59e546d090ace36600e29032, user.id=ecf4db289fba4e4a806198c62d31f7c4) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:37.183137 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5f3cb2-a5fb-43b7-8600-4d810b86bdf3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:37.184833 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5f3cb2-a5fb-43b7-8600-4d810b86bdf3 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:37.199003 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 349/680] 10.222.0.81 () {56 vars in 1374 bytes} [Wed Jul 20 22:26:37 2022] GET /identity/v3/role_assignments?user.id=ecf4db289fba4e4a806198c62d31f7c4&scope.project.id=385edf2e59e546d090ace36600e29032&role.id=9b4a83e2eca44a8da9f356bc40f8788a => generated 608 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:37.207125 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db82d9f0-2fff-4b3e-b273-f367067f6158 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:37.207878 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db82d9f0-2fff-4b3e-b273-f367067f6158 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:37.208369 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db82d9f0-2fff-4b3e-b273-f367067f6158 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:37.210356 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 350/681] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:26:37 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:26:37.215837 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2dc1d1-4fcd-47da-ba2c-f950523dd44f None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:37.216437 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2dc1d1-4fcd-47da-ba2c-f950523dd44f None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:37.216935 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2dc1d1-4fcd-47da-ba2c-f950523dd44f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:37.231498 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:37.245997 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da022219-761f-4cdb-9357-373632b34956 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:37.260615 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-7d2dc1d1-4fcd-47da-ba2c-f950523dd44f None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:26:37.261841 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-7d2dc1d1-4fcd-47da-ba2c-f950523dd44f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:26:37.321531 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1155d5ab-1356-4a8f-a127-ccb8c7723f62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:37.322443 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1155d5ab-1356-4a8f-a127-ccb8c7723f62 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:37.322735 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1155d5ab-1356-4a8f-a127-ccb8c7723f62 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:37.323032 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1155d5ab-1356-4a8f-a127-ccb8c7723f62 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:37.335650 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1155d5ab-1356-4a8f-a127-ccb8c7723f62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:37.337134 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1155d5ab-1356-4a8f-a127-ccb8c7723f62 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:37.358789 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-1155d5ab-1356-4a8f-a127-ccb8c7723f62 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:26:37.358789 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:37.358789 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:37.358789 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:37.358789 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:37.358789 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:37.358789 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:37.358789 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:37.358789 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:37.358789 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:37.358789 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:37.358789 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:37.358789 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 22:26:37.358789 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 22:26:37.358789 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 22:26:37.359801 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 22:26:37.359801 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:37.359801 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:37.359801 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:37.359801 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:37.359801 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:37.359801 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:37.359801 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:37.359801 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:37.359801 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:37.359801 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:37.359801 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:37.359801 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:37.359801 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:37.359801 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:37.359801 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:37.359801 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:37.360742 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 22:26:37.360742 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 22:26:37.360742 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:26:37.360742 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:26:37.360742 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 332/682] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Jul 20 22:26:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:37.367270 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1155d5ab-1356-4a8f-a127-ccb8c7723f62 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:37.402039 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 351/683] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:26:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 22:26:37.438696 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2dc1d1-4fcd-47da-ba2c-f950523dd44f None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:37.477992 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:37.512404 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1155d5ab-1356-4a8f-a127-ccb8c7723f62 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:37.556536 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1a27ba-8c1e-4cb6-96be-393b71f9e0b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:37.557456 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1a27ba-8c1e-4cb6-96be-393b71f9e0b1 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:37.557888 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1a27ba-8c1e-4cb6-96be-393b71f9e0b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:37.558337 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1a27ba-8c1e-4cb6-96be-393b71f9e0b1 None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:37.559255 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1a27ba-8c1e-4cb6-96be-393b71f9e0b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:37.559819 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1a27ba-8c1e-4cb6-96be-393b71f9e0b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:37.561377 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1a27ba-8c1e-4cb6-96be-393b71f9e0b1 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:37.570296 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:37.576641 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 333/684] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Jul 20 22:26:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:37.577014 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2dc1d1-4fcd-47da-ba2c-f950523dd44f None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:37.585441 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1a27ba-8c1e-4cb6-96be-393b71f9e0b1 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:37.618759 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0b7fbb-4dea-4463-b71d-366fdc27e249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:37.619765 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0b7fbb-4dea-4463-b71d-366fdc27e249 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:37.620050 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0b7fbb-4dea-4463-b71d-366fdc27e249 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:37.620328 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0b7fbb-4dea-4463-b71d-366fdc27e249 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:37.635295 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0b7fbb-4dea-4463-b71d-366fdc27e249 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 22:26:37.644285 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-6f0b7fbb-4dea-4463-b71d-366fdc27e249 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 22:26:37.644285 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:37.644285 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:37.644285 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:37.644285 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:37.644285 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:37.644285 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:37.644285 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:37.644285 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:37.644285 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:37.644285 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:37.644285 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:37.644285 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 22:26:37.644285 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 22:26:37.644285 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 22:26:37.644285 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 22:26:37.645417 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 22:26:37.645417 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 22:26:37.645417 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 22:26:37.645417 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 22:26:37.645417 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:37.645417 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:37.645417 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:37.645417 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:37.645417 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:37.645417 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:37.645417 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:37.645417 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:37.645417 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:37.645417 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:37.645417 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:37.645417 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:37.646239 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:37.646239 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:37.646239 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:37.646239 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:37.646239 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 22:26:37.646239 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 22:26:37.646239 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 22:26:37.646239 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 22:26:37.646239 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 22:26:37.646239 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 22:26:37.646239 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 22:26:37.646239 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:26:37.646239 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 352/685] 10.222.0.81 () {56 vars in 1096 bytes} [Wed Jul 20 22:26:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 22:26:37.656154 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0b7fbb-4dea-4463-b71d-366fdc27e249 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:37.711376 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1a27ba-8c1e-4cb6-96be-393b71f9e0b1 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:37.760451 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965aa411-5dde-4001-8d58-8c36b6ea991a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:37.763004 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965aa411-5dde-4001-8d58-8c36b6ea991a None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:37.764322 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965aa411-5dde-4001-8d58-8c36b6ea991a None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:37.764837 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965aa411-5dde-4001-8d58-8c36b6ea991a None admin] PATH_INFO: `/v3/users/swift` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:37.776210 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965aa411-5dde-4001-8d58-8c36b6ea991a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:37.777773 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965aa411-5dde-4001-8d58-8c36b6ea991a None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:37.783766 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:37.792360 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0b7fbb-4dea-4463-b71d-366fdc27e249 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:37.807064 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-965aa411-5dde-4001-8d58-8c36b6ea991a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 20 22:26:37.807064 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:37.807064 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:37.807064 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:37.807064 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:37.807064 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:37.807064 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:37.807064 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:37.807064 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:37.807064 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:37.807064 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:37.807064 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:37.807064 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 22:26:37.807064 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 22:26:37.807064 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 22:26:37.807064 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 22:26:37.808069 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:37.808069 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:37.808069 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 22:26:37.808069 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 22:26:37.808069 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 22:26:37.808069 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 22:26:37.808069 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:37.808069 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:37.808069 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:37.808069 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:37.808069 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:37.808069 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:37.808069 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:37.808069 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:37.808069 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:37.808069 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:37.808069 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:37.808069 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:37.809010 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:37.809010 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:37.809010 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 22:26:37.809010 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 22:26:37.809010 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 22:26:37.809010 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 22:26:37.809010 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 22:26:37.809010 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 22:26:37.809010 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 20 22:26:37.809010 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:26:37.809010 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 334/686] 10.222.0.81 () {56 vars in 1096 bytes} [Wed Jul 20 22:26:37 2022] GET /identity/v3/users/swift => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:37.815104 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965aa411-5dde-4001-8d58-8c36b6ea991a None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:37.861046 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4189ff-a7c7-463e-8686-47a43d0a16db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:37.861731 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4189ff-a7c7-463e-8686-47a43d0a16db None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:37.862015 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4189ff-a7c7-463e-8686-47a43d0a16db None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:37.862294 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4189ff-a7c7-463e-8686-47a43d0a16db None admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:37.863089 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4189ff-a7c7-463e-8686-47a43d0a16db None admin] RBAC: Adding query filter params (name=admin) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:37.863433 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4189ff-a7c7-463e-8686-47a43d0a16db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:37.865326 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4189ff-a7c7-463e-8686-47a43d0a16db None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:37.886661 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 353/687] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Jul 20 22:26:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:37.895419 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4189ff-a7c7-463e-8686-47a43d0a16db None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:37.916189 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:37.968667 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965aa411-5dde-4001-8d58-8c36b6ea991a None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:38.023875 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d905e6c-898b-4fad-80ab-6913c829b541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:38.024491 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d905e6c-898b-4fad-80ab-6913c829b541 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:38.024783 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d905e6c-898b-4fad-80ab-6913c829b541 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:38.026708 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d905e6c-898b-4fad-80ab-6913c829b541 None admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:38.027583 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d905e6c-898b-4fad-80ab-6913c829b541 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:38.027900 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d905e6c-898b-4fad-80ab-6913c829b541 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:38.029333 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d905e6c-898b-4fad-80ab-6913c829b541 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:38.065268 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4189ff-a7c7-463e-8686-47a43d0a16db None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:38.076855 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 335/688] 10.222.0.81 () {56 vars in 1135 bytes} [Wed Jul 20 22:26:37 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 22:26:38.083233 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d905e6c-898b-4fad-80ab-6913c829b541 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:38.137150 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838447a9-5a65-4d90-a053-e5fe0f25f28c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:38.138363 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838447a9-5a65-4d90-a053-e5fe0f25f28c None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:38.139696 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838447a9-5a65-4d90-a053-e5fe0f25f28c None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:38.140894 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838447a9-5a65-4d90-a053-e5fe0f25f28c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:38.162497 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838447a9-5a65-4d90-a053-e5fe0f25f28c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:38.165270 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838447a9-5a65-4d90-a053-e5fe0f25f28c None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:38.201555 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-838447a9-5a65-4d90-a053-e5fe0f25f28c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:26:38.201555 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:38.201555 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:38.201555 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:38.201555 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:38.201555 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:38.201555 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:38.201555 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:38.201555 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:38.201555 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:38.201555 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:38.201555 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:38.201555 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 22:26:38.201555 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 22:26:38.201555 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 22:26:38.202416 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 22:26:38.202416 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:38.202416 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:38.202416 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:38.202416 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:38.202416 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:38.202416 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:38.202416 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:38.202416 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:38.202416 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:38.202416 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:38.202416 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:38.202416 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:38.202416 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:38.202416 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:38.202416 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:38.202416 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:38.203326 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 22:26:38.203326 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 22:26:38.203326 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:26:38.203326 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:26:38.211771 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 354/689] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Jul 20 22:26:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 22:26:38.215311 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838447a9-5a65-4d90-a053-e5fe0f25f28c None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:38.240987 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:38.282374 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d905e6c-898b-4fad-80ab-6913c829b541 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:38.321046 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:38.344929 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01849cad-81c6-4357-9e51-b2c83bf4999c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:38.345134 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01849cad-81c6-4357-9e51-b2c83bf4999c None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:38.345438 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01849cad-81c6-4357-9e51-b2c83bf4999c None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:38.345633 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01849cad-81c6-4357-9e51-b2c83bf4999c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:38.379344 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838447a9-5a65-4d90-a053-e5fe0f25f28c None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:38.415731 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1256c25d-6059-4a73-ac3e-85220469edd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:38.416818 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1256c25d-6059-4a73-ac3e-85220469edd5 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:38.417348 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1256c25d-6059-4a73-ac3e-85220469edd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:38.417826 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1256c25d-6059-4a73-ac3e-85220469edd5 None admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:38.418764 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1256c25d-6059-4a73-ac3e-85220469edd5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:38.419312 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1256c25d-6059-4a73-ac3e-85220469edd5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:38.421015 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1256c25d-6059-4a73-ac3e-85220469edd5 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:38.423019 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01849cad-81c6-4357-9e51-b2c83bf4999c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:38.424612 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01849cad-81c6-4357-9e51-b2c83bf4999c None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:38.434970 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 355/690] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Jul 20 22:26:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:38.440763 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b0cd71-3efc-4ffd-b0df-57b44bbfa63c None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:38.441846 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b0cd71-3efc-4ffd-b0df-57b44bbfa63c None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:38.442344 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b0cd71-3efc-4ffd-b0df-57b44bbfa63c None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:38.444124 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 356/691] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:26:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:26:38.445554 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b0cd71-3efc-4ffd-b0df-57b44bbfa63c None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:38.515104 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:38.537612 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 336/692] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Jul 20 22:26:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 20 22:26:38.539828 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61d83e9-95c1-4f48-b865-98be1728fd02 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:38.540306 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61d83e9-95c1-4f48-b865-98be1728fd02 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:38.540994 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61d83e9-95c1-4f48-b865-98be1728fd02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:38.551906 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b0cd71-3efc-4ffd-b0df-57b44bbfa63c None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:38.558475 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-a61d83e9-95c1-4f48-b865-98be1728fd02 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:26:38.560847 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-a61d83e9-95c1-4f48-b865-98be1728fd02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:26:38.588977 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f5490f-2760-4f5d-a89b-7deed15cf334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:38.589947 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f5490f-2760-4f5d-a89b-7deed15cf334 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:38.590307 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f5490f-2760-4f5d-a89b-7deed15cf334 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:38.590634 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f5490f-2760-4f5d-a89b-7deed15cf334 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:38.599256 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f5490f-2760-4f5d-a89b-7deed15cf334 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:38.599379 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:38.601304 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f5490f-2760-4f5d-a89b-7deed15cf334 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:38.616431 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-78f5490f-2760-4f5d-a89b-7deed15cf334 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 20 22:26:38.616431 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:38.616431 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:38.616431 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:38.616431 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:38.616431 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:38.616431 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:38.616431 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:38.616431 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:38.616431 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:38.616431 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:38.616431 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:38.616431 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 22:26:38.616431 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 22:26:38.616431 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 22:26:38.616431 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 22:26:38.617394 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:38.617394 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:38.617394 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 22:26:38.617394 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 22:26:38.617394 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 22:26:38.617394 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 22:26:38.617394 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:38.617394 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:38.617394 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:38.617394 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:38.617394 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:38.617394 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:38.617394 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:38.617394 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:38.617394 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:38.617394 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:38.617394 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:38.617394 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:38.618328 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:38.618328 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:38.618328 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 22:26:38.618328 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 22:26:38.618328 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 22:26:38.618328 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 22:26:38.618328 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 22:26:38.618328 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 22:26:38.618328 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 20 22:26:38.618328 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:26:38.622754 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 357/693] 10.222.0.81 () {56 vars in 1093 bytes} [Wed Jul 20 22:26:38 2022] GET /identity/v3/users/nova => generated 82 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:38.623680 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f5490f-2760-4f5d-a89b-7deed15cf334 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:38.642854 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 337/694] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:26:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 22:26:38.644153 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61d83e9-95c1-4f48-b865-98be1728fd02 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:38.734963 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f5490f-2760-4f5d-a89b-7deed15cf334 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:38.764428 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:38.773195 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30209ac3-9859-4ee3-9b22-7e9df9c5d78f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:38.780275 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30209ac3-9859-4ee3-9b22-7e9df9c5d78f None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:38.780275 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30209ac3-9859-4ee3-9b22-7e9df9c5d78f None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:38.780480 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61d83e9-95c1-4f48-b865-98be1728fd02 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:38.780557 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30209ac3-9859-4ee3-9b22-7e9df9c5d78f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:38.787256 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30209ac3-9859-4ee3-9b22-7e9df9c5d78f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:38.789173 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30209ac3-9859-4ee3-9b22-7e9df9c5d78f None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:38.809499 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-30209ac3-9859-4ee3-9b22-7e9df9c5d78f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:26:38.809499 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:38.809499 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:38.809499 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:38.809499 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:38.809499 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:38.809499 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:38.809499 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:38.809499 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:38.809499 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:38.809499 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:38.809499 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:38.809499 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 22:26:38.809499 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 22:26:38.809499 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 22:26:38.810453 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 22:26:38.810453 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:38.810453 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:38.810453 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:38.810453 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:38.810453 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:38.810453 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:38.810453 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:38.810453 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:38.810453 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:38.810453 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:38.810453 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:38.810453 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:38.810453 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:38.810453 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:38.810453 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:38.810453 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:38.811849 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 22:26:38.811849 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 22:26:38.811849 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:26:38.811849 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:26:38.811849 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 358/695] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Jul 20 22:26:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:38.812175 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30209ac3-9859-4ee3-9b22-7e9df9c5d78f None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:38.820574 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e272d03f-9a72-4ab1-93d8-77668b724c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:38.821594 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e272d03f-9a72-4ab1-93d8-77668b724c76 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:38.822088 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e272d03f-9a72-4ab1-93d8-77668b724c76 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:38.822637 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e272d03f-9a72-4ab1-93d8-77668b724c76 None admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:38.823868 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e272d03f-9a72-4ab1-93d8-77668b724c76 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:38.824497 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e272d03f-9a72-4ab1-93d8-77668b724c76 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:38.826413 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e272d03f-9a72-4ab1-93d8-77668b724c76 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:38.846568 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 338/696] 10.222.0.81 () {56 vars in 1133 bytes} [Wed Jul 20 22:26:38 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:38.849429 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e272d03f-9a72-4ab1-93d8-77668b724c76 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:38.865010 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:38.928838 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30209ac3-9859-4ee3-9b22-7e9df9c5d78f None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:38.961415 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e272d03f-9a72-4ab1-93d8-77668b724c76 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:38.965898 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a74fb6-44b1-4c15-9deb-07839339edcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:38.966702 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a74fb6-44b1-4c15-9deb-07839339edcb None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:38.966982 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a74fb6-44b1-4c15-9deb-07839339edcb None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:38.967295 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a74fb6-44b1-4c15-9deb-07839339edcb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:38.978832 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a74fb6-44b1-4c15-9deb-07839339edcb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:38.980342 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a74fb6-44b1-4c15-9deb-07839339edcb None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:38.999704 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-43a74fb6-44b1-4c15-9deb-07839339edcb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:26:38.999704 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:38.999704 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:38.999704 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:38.999704 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:38.999704 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:38.999704 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:38.999704 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:38.999704 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:38.999704 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:38.999704 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:38.999704 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:38.999704 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 22:26:38.999704 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 22:26:38.999704 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 22:26:39.000617 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 22:26:39.000617 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:39.000617 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:39.000617 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:39.000617 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:39.000617 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:39.000617 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:39.000617 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:39.000617 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:39.000617 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:39.000617 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:39.000617 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:39.000617 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:39.000617 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:39.000617 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:39.000617 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:39.000617 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:39.001506 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 22:26:39.001506 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 22:26:39.001506 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:26:39.001506 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:26:39.002644 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0536524-ca40-4d86-942a-2eab78ce7556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:39.003615 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0536524-ca40-4d86-942a-2eab78ce7556 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:39.004652 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0536524-ca40-4d86-942a-2eab78ce7556 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:39.005147 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0536524-ca40-4d86-942a-2eab78ce7556 None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:39.006070 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0536524-ca40-4d86-942a-2eab78ce7556 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:39.006581 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 359/697] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Jul 20 22:26:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:39.006787 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0536524-ca40-4d86-942a-2eab78ce7556 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:39.007447 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a74fb6-44b1-4c15-9deb-07839339edcb None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:39.008598 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0536524-ca40-4d86-942a-2eab78ce7556 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:39.022230 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 339/698] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Jul 20 22:26:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:39.028082 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0536524-ca40-4d86-942a-2eab78ce7556 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:39.042579 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:39.128067 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:39.164448 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a74fb6-44b1-4c15-9deb-07839339edcb None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:39.217913 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0536524-ca40-4d86-942a-2eab78ce7556 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:39.224908 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9445719-b211-42f6-98f1-50b397569ad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:39.226057 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9445719-b211-42f6-98f1-50b397569ad3 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:39.226591 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9445719-b211-42f6-98f1-50b397569ad3 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:39.227091 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9445719-b211-42f6-98f1-50b397569ad3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:39.278053 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95985c06-816a-46f8-a9f9-8a649f7f8843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:39.279749 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95985c06-816a-46f8-a9f9-8a649f7f8843 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:39.280490 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95985c06-816a-46f8-a9f9-8a649f7f8843 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:39.281377 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95985c06-816a-46f8-a9f9-8a649f7f8843 None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:39.282913 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95985c06-816a-46f8-a9f9-8a649f7f8843 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:39.283709 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95985c06-816a-46f8-a9f9-8a649f7f8843 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:39.286538 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95985c06-816a-46f8-a9f9-8a649f7f8843 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:39.309946 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 340/699] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Jul 20 22:26:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:39.311783 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95985c06-816a-46f8-a9f9-8a649f7f8843 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:39.357345 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9445719-b211-42f6-98f1-50b397569ad3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:39.359052 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9445719-b211-42f6-98f1-50b397569ad3 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:39.394136 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:39.444250 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95985c06-816a-46f8-a9f9-8a649f7f8843 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:39.470310 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 360/700] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Jul 20 22:26:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 22:26:39.471638 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9445719-b211-42f6-98f1-50b397569ad3 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:39.482528 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:39.495406 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a992ca2e-9341-46d2-bc18-1a74017b9d64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:39.496469 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a992ca2e-9341-46d2-bc18-1a74017b9d64 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:39.496825 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a992ca2e-9341-46d2-bc18-1a74017b9d64 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:39.497168 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a992ca2e-9341-46d2-bc18-1a74017b9d64 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:39.504792 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a992ca2e-9341-46d2-bc18-1a74017b9d64 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:39.506958 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a992ca2e-9341-46d2-bc18-1a74017b9d64 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:39.521438 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-a992ca2e-9341-46d2-bc18-1a74017b9d64 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 22:26:39.521438 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:39.521438 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:39.521438 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:39.521438 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:39.521438 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:39.521438 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:39.521438 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:39.521438 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:39.521438 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:39.521438 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:39.521438 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:39.521438 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 22:26:39.521438 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 22:26:39.521438 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 22:26:39.522517 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 22:26:39.522517 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:39.522517 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:39.522517 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:39.522517 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:39.522517 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:39.522517 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:39.522517 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:39.522517 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:39.522517 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:39.522517 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:39.522517 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:39.522517 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:39.522517 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:39.522517 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:39.522517 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:39.522517 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:39.523804 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 22:26:39.523804 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 22:26:39.523804 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 22:26:39.523804 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 22:26:39.523804 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 22:26:39.523804 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 22:26:39.523804 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 22:26:39.523804 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:26:39.523804 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 341/701] 10.222.0.81 () {56 vars in 1111 bytes} [Wed Jul 20 22:26:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:39.528462 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f814ed7-748c-43c4-abe1-7a1cf64e58f2 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:39.529256 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f814ed7-748c-43c4-abe1-7a1cf64e58f2 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:39.529784 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f814ed7-748c-43c4-abe1-7a1cf64e58f2 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:39.531270 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 342/702] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:26:39 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:26:39.532107 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f814ed7-748c-43c4-abe1-7a1cf64e58f2 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:39.591836 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9445719-b211-42f6-98f1-50b397569ad3 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:39.624595 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:39.629996 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8edd15-b0a5-4788-a1cd-afea848a50df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:39.630688 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8edd15-b0a5-4788-a1cd-afea848a50df None admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:39.630969 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8edd15-b0a5-4788-a1cd-afea848a50df None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:39.631350 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8edd15-b0a5-4788-a1cd-afea848a50df None admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:39.632078 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8edd15-b0a5-4788-a1cd-afea848a50df None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:39.633584 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8edd15-b0a5-4788-a1cd-afea848a50df None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:39.642617 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f814ed7-748c-43c4-abe1-7a1cf64e58f2 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:39.676826 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d5d620-461f-4e66-8c75-a45fe7d19c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:39.677645 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d5d620-461f-4e66-8c75-a45fe7d19c9c None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:39.677929 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d5d620-461f-4e66-8c75-a45fe7d19c9c None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:39.678204 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d5d620-461f-4e66-8c75-a45fe7d19c9c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:39.688582 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d5d620-461f-4e66-8c75-a45fe7d19c9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:39.690013 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d5d620-461f-4e66-8c75-a45fe7d19c9c None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:39.707793 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-95d5d620-461f-4e66-8c75-a45fe7d19c9c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:26:39.707793 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:39.707793 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:39.707793 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:39.707793 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:39.707793 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:39.707793 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:39.707793 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:39.707793 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:39.707793 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:39.707793 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:39.707793 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:39.707793 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 22:26:39.707793 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 22:26:39.707793 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 22:26:39.716471 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 22:26:39.716471 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:39.716471 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:39.716471 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:39.716471 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:39.716471 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:39.716471 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:39.716471 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:39.716471 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:39.716471 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:39.716471 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:39.716471 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:39.716471 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:39.716471 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:39.716471 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:39.716471 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:39.716471 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:39.717564 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 22:26:39.717564 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 22:26:39.717564 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:26:39.717564 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:26:39.717564 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 343/703] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Jul 20 22:26:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:39.717564 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d5d620-461f-4e66-8c75-a45fe7d19c9c None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:39.746181 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:39.807895 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 361/704] 10.222.0.81 () {60 vars in 1132 bytes} [Wed Jul 20 22:26:39 2022] POST /identity/v3/users => generated 254 bytes in 337 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:26:39.810027 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d7871a-c7d8-4073-bae3-e146cd038cc7 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:39.810533 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d7871a-c7d8-4073-bae3-e146cd038cc7 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:39.811004 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d7871a-c7d8-4073-bae3-e146cd038cc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:39.811767 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d5d620-461f-4e66-8c75-a45fe7d19c9c None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:39.837168 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-62d7871a-c7d8-4073-bae3-e146cd038cc7 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:26:39.838980 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-62d7871a-c7d8-4073-bae3-e146cd038cc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:26:39.857881 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6daf3d-669a-4992-ac83-b4fd34a8b7ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:39.858556 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6daf3d-669a-4992-ac83-b4fd34a8b7ef None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:39.858849 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6daf3d-669a-4992-ac83-b4fd34a8b7ef None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:39.859318 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6daf3d-669a-4992-ac83-b4fd34a8b7ef None admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:39.860561 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6daf3d-669a-4992-ac83-b4fd34a8b7ef None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:39.860977 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6daf3d-669a-4992-ac83-b4fd34a8b7ef None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:39.862443 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6daf3d-669a-4992-ac83-b4fd34a8b7ef None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:39.887521 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 344/705] 10.222.0.81 () {56 vars in 1148 bytes} [Wed Jul 20 22:26:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:39.887521 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6daf3d-669a-4992-ac83-b4fd34a8b7ef None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:39.934651 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 362/706] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:26:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 22:26:39.936077 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e28595-a737-4a32-8e5a-e00c7929a947 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:39.936373 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e28595-a737-4a32-8e5a-e00c7929a947 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:39.936664 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e28595-a737-4a32-8e5a-e00c7929a947 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:39.938160 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 363/707] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:26:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:26:39.943038 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e28595-a737-4a32-8e5a-e00c7929a947 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:40.017916 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:40.026600 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6daf3d-669a-4992-ac83-b4fd34a8b7ef None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:40.060276 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e28595-a737-4a32-8e5a-e00c7929a947 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:40.064447 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb0b0b3-387d-44bc-bfa9-b2f27d66ec0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:40.065131 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb0b0b3-387d-44bc-bfa9-b2f27d66ec0d None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:40.065421 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb0b0b3-387d-44bc-bfa9-b2f27d66ec0d None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:40.065762 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb0b0b3-387d-44bc-bfa9-b2f27d66ec0d None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:40.067085 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb0b0b3-387d-44bc-bfa9-b2f27d66ec0d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:40.067402 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb0b0b3-387d-44bc-bfa9-b2f27d66ec0d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:40.068877 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb0b0b3-387d-44bc-bfa9-b2f27d66ec0d None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:40.080833 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:40.085584 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 345/708] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Jul 20 22:26:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:40.085584 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab05fc1-002d-4a44-9dfd-0daf10dde8b1 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:40.085584 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab05fc1-002d-4a44-9dfd-0daf10dde8b1 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:40.085584 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab05fc1-002d-4a44-9dfd-0daf10dde8b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:40.100788 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6872e3de-7a8b-4b20-895a-a790aae142e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:40.103054 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6872e3de-7a8b-4b20-895a-a790aae142e2 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:40.103054 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6872e3de-7a8b-4b20-895a-a790aae142e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:40.103054 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6872e3de-7a8b-4b20-895a-a790aae142e2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:40.103355 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-fab05fc1-002d-4a44-9dfd-0daf10dde8b1 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:26:40.103451 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6872e3de-7a8b-4b20-895a-a790aae142e2 None admin] RBAC: Adding query filter params (role.id=9b4a83e2eca44a8da9f356bc40f8788a, scope.project.id=385edf2e59e546d090ace36600e29032, user.id=70e97436a08543a180225561ef8bc817) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:40.103451 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6872e3de-7a8b-4b20-895a-a790aae142e2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:40.103723 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-fab05fc1-002d-4a44-9dfd-0daf10dde8b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:26:40.104674 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6872e3de-7a8b-4b20-895a-a790aae142e2 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:40.111092 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 364/709] 10.222.0.81 () {56 vars in 1374 bytes} [Wed Jul 20 22:26:39 2022] GET /identity/v3/role_assignments?user.id=70e97436a08543a180225561ef8bc817&scope.project.id=385edf2e59e546d090ace36600e29032&role.id=9b4a83e2eca44a8da9f356bc40f8788a => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:40.111697 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6872e3de-7a8b-4b20-895a-a790aae142e2 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:40.217108 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 346/710] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:26:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 22:26:40.217983 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab05fc1-002d-4a44-9dfd-0daf10dde8b1 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:40.269721 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6872e3de-7a8b-4b20-895a-a790aae142e2 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:40.318339 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f70a388-5810-4700-ab02-206a6c32322a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:40.319824 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f70a388-5810-4700-ab02-206a6c32322a None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:40.320101 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f70a388-5810-4700-ab02-206a6c32322a None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:40.320373 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f70a388-5810-4700-ab02-206a6c32322a None admin] PATH_INFO: `/v3/services/network` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:40.321024 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f70a388-5810-4700-ab02-206a6c32322a None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:40.325964 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f70a388-5810-4700-ab02-206a6c32322a None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:40.333664 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:40.342550 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-6f70a388-5810-4700-ab02-206a6c32322a None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 20 22:26:40.342550 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:40.342550 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 22:26:40.342550 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 22:26:40.342550 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 22:26:40.342550 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 22:26:40.342550 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 22:26:40.342550 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 22:26:40.342550 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 20 22:26:40.342550 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:26:40.342550 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 22:26:40.342550 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:26:40.342550 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:40.342550 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:40.342550 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:40.342550 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:40.342550 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:40.343891 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:40.343891 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:40.343891 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:40.343891 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:40.343891 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:40.343891 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:40.343891 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 22:26:40.343891 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 22:26:40.343891 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 22:26:40.343891 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 22:26:40.343891 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:40.343891 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:40.343891 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:40.343891 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:40.343891 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:40.343891 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:40.344926 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:40.344926 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:40.344926 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:40.344926 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:40.344926 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:40.344926 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:40.344926 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:40.344926 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:40.344926 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:40.344926 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:40.344926 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 22:26:40.344926 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 22:26:40.344926 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 20 22:26:40.344926 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:26:40.348753 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 365/711] 10.222.0.81 () {56 vars in 1111 bytes} [Wed Jul 20 22:26:40 2022] GET /identity/v3/services/network => generated 88 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:40.349638 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f70a388-5810-4700-ab02-206a6c32322a None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:40.353191 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab05fc1-002d-4a44-9dfd-0daf10dde8b1 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:40.396838 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c603ca-8f4f-40ac-bc8c-96f452329e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:40.397034 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c603ca-8f4f-40ac-bc8c-96f452329e6f None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:40.397410 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c603ca-8f4f-40ac-bc8c-96f452329e6f None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:40.397753 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c603ca-8f4f-40ac-bc8c-96f452329e6f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:40.409892 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c603ca-8f4f-40ac-bc8c-96f452329e6f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:40.410631 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c603ca-8f4f-40ac-bc8c-96f452329e6f None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:40.411410 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:40.429104 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-34c603ca-8f4f-40ac-bc8c-96f452329e6f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 22:26:40.429104 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:40.429104 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:40.429104 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:40.429104 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:40.429104 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:40.429104 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:40.429104 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:40.429104 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:40.429104 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:40.429104 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:40.429104 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:40.429104 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 22:26:40.429104 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 22:26:40.429104 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 22:26:40.430207 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 22:26:40.430207 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:40.430207 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:40.430207 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:40.430207 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:40.430207 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:40.430207 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:40.430207 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:40.430207 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:40.430207 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:40.430207 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:40.430207 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:40.430207 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:40.430207 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:40.430207 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:40.430207 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:40.430207 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:40.431468 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 22:26:40.431468 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 22:26:40.431468 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 22:26:40.431468 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 22:26:40.431468 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 22:26:40.431468 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 22:26:40.431468 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 22:26:40.431468 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:26:40.431971 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 347/712] 10.222.0.81 () {56 vars in 1111 bytes} [Wed Jul 20 22:26:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:40.435076 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c603ca-8f4f-40ac-bc8c-96f452329e6f None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:40.482469 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f70a388-5810-4700-ab02-206a6c32322a None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:40.535522 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45de27b-1b37-45b5-97e1-75f15f5c7910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:40.537110 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45de27b-1b37-45b5-97e1-75f15f5c7910 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:40.537390 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45de27b-1b37-45b5-97e1-75f15f5c7910 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:40.537663 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45de27b-1b37-45b5-97e1-75f15f5c7910 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:40.539073 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45de27b-1b37-45b5-97e1-75f15f5c7910 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:40.540887 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45de27b-1b37-45b5-97e1-75f15f5c7910 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:40.558702 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-b45de27b-1b37-45b5-97e1-75f15f5c7910 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 20 22:26:40.558702 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:40.558702 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 22:26:40.558702 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 22:26:40.558702 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 22:26:40.558702 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 22:26:40.558702 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 22:26:40.558702 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 22:26:40.558702 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 20 22:26:40.558702 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:26:40.558702 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 22:26:40.558702 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:26:40.558702 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:40.558702 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:40.558702 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:40.558702 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:40.559705 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:40.559705 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:40.559705 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:40.559705 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:40.559705 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:40.559705 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:40.559705 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:40.559705 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 22:26:40.559705 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 22:26:40.559705 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 22:26:40.559705 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 22:26:40.559705 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:40.559705 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:40.559705 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:40.559705 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:40.559705 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:40.560535 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:40.560535 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:40.560535 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:40.560535 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:40.560535 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:40.560535 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:40.560535 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:40.560535 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:40.560535 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:40.560535 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:40.560535 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:40.560535 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 22:26:40.560535 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 22:26:40.560535 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 20 22:26:40.560535 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:26:40.561320 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 366/713] 10.222.0.81 () {56 vars in 1129 bytes} [Wed Jul 20 22:26:40 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:40.561890 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45de27b-1b37-45b5-97e1-75f15f5c7910 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:40.568383 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c603ca-8f4f-40ac-bc8c-96f452329e6f None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:40.609421 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b82e49-bae8-46e8-b56e-06bad59039aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:40.610131 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b82e49-bae8-46e8-b56e-06bad59039aa None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:40.610417 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b82e49-bae8-46e8-b56e-06bad59039aa None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:40.610697 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b82e49-bae8-46e8-b56e-06bad59039aa None admin] PATH_INFO: `/v3/services` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:40.611693 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b82e49-bae8-46e8-b56e-06bad59039aa None admin] RBAC: Adding query filter params (name=network) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:40.612006 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b82e49-bae8-46e8-b56e-06bad59039aa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:40.614378 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b82e49-bae8-46e8-b56e-06bad59039aa None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:40.620394 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 348/714] 10.222.0.81 () {56 vars in 1112 bytes} [Wed Jul 20 22:26:40 2022] GET /identity/v3/services?name=network => generated 124 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:40.621432 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b82e49-bae8-46e8-b56e-06bad59039aa None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:40.638869 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:40.700932 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45de27b-1b37-45b5-97e1-75f15f5c7910 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:40.714373 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:40.739494 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b82e49-bae8-46e8-b56e-06bad59039aa None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:40.740082 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa43f87-77d2-4844-867a-b23182edbcae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:40.740558 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa43f87-77d2-4844-867a-b23182edbcae None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:40.740848 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa43f87-77d2-4844-867a-b23182edbcae None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:40.741128 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa43f87-77d2-4844-867a-b23182edbcae None admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:40.741923 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa43f87-77d2-4844-867a-b23182edbcae None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:40.742347 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa43f87-77d2-4844-867a-b23182edbcae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:40.744014 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa43f87-77d2-4844-867a-b23182edbcae None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:40.762130 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 367/715] 10.222.0.81 () {56 vars in 1148 bytes} [Wed Jul 20 22:26:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:40.762130 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa43f87-77d2-4844-867a-b23182edbcae None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:40.792815 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d45162-c80a-4fde-affe-350464873208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:40.794139 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d45162-c80a-4fde-affe-350464873208 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:40.794806 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d45162-c80a-4fde-affe-350464873208 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:40.795272 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d45162-c80a-4fde-affe-350464873208 None admin] PATH_INFO: `/v3/services` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:40.796368 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d45162-c80a-4fde-affe-350464873208 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:40.797402 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d45162-c80a-4fde-affe-350464873208 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:40.799198 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d45162-c80a-4fde-affe-350464873208 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:40.806898 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 349/716] 10.222.0.81 () {56 vars in 1124 bytes} [Wed Jul 20 22:26:40 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:40.807893 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d45162-c80a-4fde-affe-350464873208 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:40.926727 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa43f87-77d2-4844-867a-b23182edbcae None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:40.975372 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:41.003567 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d45162-c80a-4fde-affe-350464873208 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:41.049221 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfbf4bd-8e10-4ad5-9696-d02e1fa02d8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:41.049904 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfbf4bd-8e10-4ad5-9696-d02e1fa02d8e None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:41.050190 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfbf4bd-8e10-4ad5-9696-d02e1fa02d8e None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:41.050515 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfbf4bd-8e10-4ad5-9696-d02e1fa02d8e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:41.051386 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfbf4bd-8e10-4ad5-9696-d02e1fa02d8e None admin] RBAC: Adding query filter params (role.id=d89b6c92d35a401a8517f1d47f6be843, scope.project.id=385edf2e59e546d090ace36600e29032, user.id=7829324db2574d38b18c749d0559177f) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:41.051706 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfbf4bd-8e10-4ad5-9696-d02e1fa02d8e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:41.053146 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfbf4bd-8e10-4ad5-9696-d02e1fa02d8e None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:41.059778 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 350/717] 10.222.0.81 () {56 vars in 1374 bytes} [Wed Jul 20 22:26:40 2022] GET /identity/v3/role_assignments?user.id=7829324db2574d38b18c749d0559177f&scope.project.id=385edf2e59e546d090ace36600e29032&role.id=d89b6c92d35a401a8517f1d47f6be843 => generated 259 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:41.060736 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfbf4bd-8e10-4ad5-9696-d02e1fa02d8e None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:41.102825 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067c0294-5c40-4deb-99ca-95a00811c1d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:41.103935 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067c0294-5c40-4deb-99ca-95a00811c1d5 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:41.104447 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067c0294-5c40-4deb-99ca-95a00811c1d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:41.104924 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067c0294-5c40-4deb-99ca-95a00811c1d5 None admin] PATH_INFO: `/v3/services` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:41.105975 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067c0294-5c40-4deb-99ca-95a00811c1d5 None admin] RBAC: Adding query filter params (type=network) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:41.106483 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067c0294-5c40-4deb-99ca-95a00811c1d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:41.108187 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067c0294-5c40-4deb-99ca-95a00811c1d5 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:41.116076 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 368/718] 10.222.0.81 () {56 vars in 1112 bytes} [Wed Jul 20 22:26:40 2022] GET /identity/v3/services?type=network => generated 124 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:41.219272 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfbf4bd-8e10-4ad5-9696-d02e1fa02d8e None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:41.272185 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9511bc95-168e-447b-97c7-43f8779cd15d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:41.272792 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9511bc95-168e-447b-97c7-43f8779cd15d None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:41.273106 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9511bc95-168e-447b-97c7-43f8779cd15d None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:41.273360 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9511bc95-168e-447b-97c7-43f8779cd15d None admin] PATH_INFO: `/v3/services` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:41.274080 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9511bc95-168e-447b-97c7-43f8779cd15d None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:41.274384 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9511bc95-168e-447b-97c7-43f8779cd15d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:41.275806 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9511bc95-168e-447b-97c7-43f8779cd15d None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:41.286066 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 351/719] 10.222.0.81 () {56 vars in 1124 bytes} [Wed Jul 20 22:26:41 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:42.298103 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0c1e45-ee72-4963-81b5-1aa61e908e7f None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:42.298638 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0c1e45-ee72-4963-81b5-1aa61e908e7f None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:42.298638 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0c1e45-ee72-4963-81b5-1aa61e908e7f None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:42.300382 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 369/720] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:26:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:26:42.305547 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae24a6c0-7355-413e-b873-4a18d993cf4a None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:42.305937 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae24a6c0-7355-413e-b873-4a18d993cf4a None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:42.306116 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae24a6c0-7355-413e-b873-4a18d993cf4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:42.311179 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:42.327401 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-ae24a6c0-7355-413e-b873-4a18d993cf4a None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:26:42.328592 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-ae24a6c0-7355-413e-b873-4a18d993cf4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:26:42.414797 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 352/721] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:26:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 22:26:42.450627 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0c1e45-ee72-4963-81b5-1aa61e908e7f None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:42.464782 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad43b03-6005-46b3-bbf5-cb4ffc5fc0e0 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:42.465067 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad43b03-6005-46b3-bbf5-cb4ffc5fc0e0 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:42.465434 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad43b03-6005-46b3-bbf5-cb4ffc5fc0e0 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:42.466955 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 353/722] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:26:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 22:26:42.471849 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dd869f-b4de-42a6-b138-147790adda03 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:42.472231 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dd869f-b4de-42a6-b138-147790adda03 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:42.472450 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dd869f-b4de-42a6-b138-147790adda03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:42.491614 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:42.503126 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-f5dd869f-b4de-42a6-b138-147790adda03 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:26:42.505033 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-f5dd869f-b4de-42a6-b138-147790adda03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:26:42.608840 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0c1e45-ee72-4963-81b5-1aa61e908e7f None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:42.622198 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 354/723] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:26:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 22:26:42.646960 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dd869f-b4de-42a6-b138-147790adda03 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:42.684843 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d688b05-102f-4675-8564-77130c1ad1c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:42.685674 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d688b05-102f-4675-8564-77130c1ad1c2 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:42.685957 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d688b05-102f-4675-8564-77130c1ad1c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:42.686233 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d688b05-102f-4675-8564-77130c1ad1c2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:42.711870 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d688b05-102f-4675-8564-77130c1ad1c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 22:26:42.726040 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-2d688b05-102f-4675-8564-77130c1ad1c2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 22:26:42.726040 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:42.726040 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:42.726040 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:42.726040 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:42.726040 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:42.726040 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:42.726040 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:42.726040 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:42.726040 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:42.726040 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:42.726040 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:42.726040 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 22:26:42.726040 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 22:26:42.726040 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 22:26:42.726040 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 22:26:42.727031 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 22:26:42.727031 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 22:26:42.727031 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 22:26:42.727031 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 22:26:42.727031 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:42.727031 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:42.727031 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:42.727031 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:42.727031 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:42.727031 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:42.727031 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:42.727031 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:42.727031 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:42.727031 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:42.727031 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:42.727031 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:42.727869 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:42.727869 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:42.727869 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:42.727869 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:42.727869 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 22:26:42.727869 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 22:26:42.727869 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 22:26:42.727869 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 22:26:42.727869 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 22:26:42.727869 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 22:26:42.727869 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 22:26:42.727869 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:26:42.729995 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 370/724] 10.222.0.81 () {56 vars in 1102 bytes} [Wed Jul 20 22:26:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 22:26:42.738107 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d688b05-102f-4675-8564-77130c1ad1c2 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:42.759353 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:42.827468 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dd869f-b4de-42a6-b138-147790adda03 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:42.880878 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8105eb79-31f0-48b8-a764-ebbba394d831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:42.881692 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8105eb79-31f0-48b8-a764-ebbba394d831 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:42.881979 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8105eb79-31f0-48b8-a764-ebbba394d831 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:42.882257 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8105eb79-31f0-48b8-a764-ebbba394d831 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:42.894245 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8105eb79-31f0-48b8-a764-ebbba394d831 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 22:26:42.906784 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-8105eb79-31f0-48b8-a764-ebbba394d831 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 22:26:42.906784 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:42.906784 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:42.906784 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:42.906784 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:42.906784 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:42.906784 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:42.906784 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:42.906784 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:42.906784 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:42.906784 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:42.906784 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:42.906784 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 22:26:42.906784 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 22:26:42.906784 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 22:26:42.906784 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 22:26:42.907829 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 22:26:42.907829 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 22:26:42.907829 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 22:26:42.907829 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 22:26:42.907829 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:42.907829 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:42.907829 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:42.907829 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:42.907829 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:42.907829 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:42.907829 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:42.907829 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:42.907829 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:42.907829 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:42.907829 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:42.907829 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:42.908681 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:42.908681 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:42.908681 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:42.908681 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:42.908681 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 22:26:42.908681 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 22:26:42.908681 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 22:26:42.908681 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 22:26:42.908681 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 22:26:42.908681 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 22:26:42.908681 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 22:26:42.908681 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:26:42.908681 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 355/725] 10.222.0.81 () {56 vars in 1096 bytes} [Wed Jul 20 22:26:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:42.915746 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8105eb79-31f0-48b8-a764-ebbba394d831 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:42.937284 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:42.943771 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d688b05-102f-4675-8564-77130c1ad1c2 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:42.984525 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024e48ed-32ed-4c6f-b038-17d9013fe87c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:42.985117 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024e48ed-32ed-4c6f-b038-17d9013fe87c None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:42.985405 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024e48ed-32ed-4c6f-b038-17d9013fe87c None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:42.985682 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024e48ed-32ed-4c6f-b038-17d9013fe87c None admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:42.986432 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024e48ed-32ed-4c6f-b038-17d9013fe87c None admin] RBAC: Adding query filter params (name=service) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:42.986737 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024e48ed-32ed-4c6f-b038-17d9013fe87c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:42.988699 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024e48ed-32ed-4c6f-b038-17d9013fe87c None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:42.997195 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 371/726] 10.222.0.81 () {56 vars in 1103 bytes} [Wed Jul 20 22:26:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 22:26:43.007829 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024e48ed-32ed-4c6f-b038-17d9013fe87c None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:43.031070 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8105eb79-31f0-48b8-a764-ebbba394d831 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:43.056284 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:43.068906 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55047490-c5d1-4a4a-97a4-8cee9392e8ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:43.069587 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55047490-c5d1-4a4a-97a4-8cee9392e8ed None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:43.069888 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55047490-c5d1-4a4a-97a4-8cee9392e8ed None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:43.070209 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55047490-c5d1-4a4a-97a4-8cee9392e8ed None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:43.070944 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55047490-c5d1-4a4a-97a4-8cee9392e8ed None admin] RBAC: Adding query filter params (name=admin) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:43.071335 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55047490-c5d1-4a4a-97a4-8cee9392e8ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:43.072792 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55047490-c5d1-4a4a-97a4-8cee9392e8ed None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:43.084369 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 356/727] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Jul 20 22:26:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:43.089915 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55047490-c5d1-4a4a-97a4-8cee9392e8ed None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:43.148374 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024e48ed-32ed-4c6f-b038-17d9013fe87c None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:43.199679 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55047490-c5d1-4a4a-97a4-8cee9392e8ed None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:43.213641 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11708163-a2ca-47e6-b2a6-1949f514432d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:43.214462 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11708163-a2ca-47e6-b2a6-1949f514432d None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:43.214745 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11708163-a2ca-47e6-b2a6-1949f514432d None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:43.215039 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11708163-a2ca-47e6-b2a6-1949f514432d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:43.237023 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69e6ae1-537a-4ff8-bf08-4ce72ac3e0ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:43.237206 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11708163-a2ca-47e6-b2a6-1949f514432d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:43.238346 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69e6ae1-537a-4ff8-bf08-4ce72ac3e0ed None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:43.238632 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69e6ae1-537a-4ff8-bf08-4ce72ac3e0ed None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:43.238904 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69e6ae1-537a-4ff8-bf08-4ce72ac3e0ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:43.239199 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11708163-a2ca-47e6-b2a6-1949f514432d None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:43.263758 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69e6ae1-537a-4ff8-bf08-4ce72ac3e0ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:43.265287 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69e6ae1-537a-4ff8-bf08-4ce72ac3e0ed None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:43.270632 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-11708163-a2ca-47e6-b2a6-1949f514432d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:26:43.270632 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:43.270632 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:43.270632 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:43.270632 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:43.270632 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:43.270632 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:43.270632 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:43.270632 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:43.270632 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:43.270632 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:43.270632 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:43.270632 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 22:26:43.270632 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 22:26:43.270632 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 22:26:43.271608 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 22:26:43.271608 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:43.271608 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:43.271608 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:43.271608 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:43.271608 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:43.271608 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:43.271608 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:43.271608 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:43.271608 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:43.271608 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:43.271608 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:43.271608 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:43.271608 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:43.271608 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:43.271608 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:43.271608 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:43.272480 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 22:26:43.272480 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 22:26:43.272480 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:26:43.272480 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:26:43.272480 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 372/728] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Jul 20 22:26:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:43.285467 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11708163-a2ca-47e6-b2a6-1949f514432d None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:43.307595 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-a69e6ae1-537a-4ff8-bf08-4ce72ac3e0ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:26:43.307595 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:43.307595 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:43.307595 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:43.307595 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:43.307595 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:43.307595 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:43.307595 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:43.307595 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:43.307595 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:43.307595 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:43.307595 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:43.307595 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 22:26:43.307595 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 22:26:43.307595 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 22:26:43.310046 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 22:26:43.310046 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:43.310046 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:43.310046 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:43.310046 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:43.310046 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:43.310046 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:43.310046 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:43.310046 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:43.310046 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:43.310046 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:43.310046 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:43.310046 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:43.310046 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:43.310046 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:43.310046 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:43.310046 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:43.312155 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 22:26:43.312155 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 22:26:43.312155 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:26:43.312155 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:26:43.312155 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 357/729] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Jul 20 22:26:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:43.316556 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:43.317806 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69e6ae1-537a-4ff8-bf08-4ce72ac3e0ed None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:43.371037 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:43.437387 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69e6ae1-537a-4ff8-bf08-4ce72ac3e0ed None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:43.479226 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a38743d-367e-4e10-82c6-d533a6a9cb20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:43.479907 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a38743d-367e-4e10-82c6-d533a6a9cb20 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:43.480195 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a38743d-367e-4e10-82c6-d533a6a9cb20 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:43.480474 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a38743d-367e-4e10-82c6-d533a6a9cb20 None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:43.481200 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a38743d-367e-4e10-82c6-d533a6a9cb20 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:43.481508 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a38743d-367e-4e10-82c6-d533a6a9cb20 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:43.482895 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a38743d-367e-4e10-82c6-d533a6a9cb20 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:43.499237 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11708163-a2ca-47e6-b2a6-1949f514432d None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:43.513578 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 358/730] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Jul 20 22:26:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:43.518668 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a38743d-367e-4e10-82c6-d533a6a9cb20 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:43.545233 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ba3556-80fe-49c2-a0a7-0b5a2faa96bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:43.545962 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ba3556-80fe-49c2-a0a7-0b5a2faa96bf None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:43.546255 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ba3556-80fe-49c2-a0a7-0b5a2faa96bf None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:43.546949 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ba3556-80fe-49c2-a0a7-0b5a2faa96bf None admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:43.547714 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ba3556-80fe-49c2-a0a7-0b5a2faa96bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:43.548024 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ba3556-80fe-49c2-a0a7-0b5a2faa96bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:43.549462 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ba3556-80fe-49c2-a0a7-0b5a2faa96bf None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:43.563887 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 373/731] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Jul 20 22:26:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:43.572375 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ba3556-80fe-49c2-a0a7-0b5a2faa96bf None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:43.687270 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:43.722671 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:43.734797 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a38743d-367e-4e10-82c6-d533a6a9cb20 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:43.736836 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ba3556-80fe-49c2-a0a7-0b5a2faa96bf None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:43.782382 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31031cfd-4b0d-440b-9472-b7ab26089e97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:43.782947 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c421cd1-cfeb-41d0-8b51-c204194c25db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:43.783184 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31031cfd-4b0d-440b-9472-b7ab26089e97 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:43.783456 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31031cfd-4b0d-440b-9472-b7ab26089e97 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:43.783734 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31031cfd-4b0d-440b-9472-b7ab26089e97 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:43.783892 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c421cd1-cfeb-41d0-8b51-c204194c25db None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:43.784149 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c421cd1-cfeb-41d0-8b51-c204194c25db None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:43.784470 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c421cd1-cfeb-41d0-8b51-c204194c25db None admin] PATH_INFO: `/v3/users/swift` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:43.792068 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31031cfd-4b0d-440b-9472-b7ab26089e97 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:43.793506 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31031cfd-4b0d-440b-9472-b7ab26089e97 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:43.793610 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c421cd1-cfeb-41d0-8b51-c204194c25db None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:43.794946 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c421cd1-cfeb-41d0-8b51-c204194c25db None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:43.808826 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-31031cfd-4b0d-440b-9472-b7ab26089e97 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 22:26:43.808826 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:43.808826 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:43.808826 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:43.808826 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:43.808826 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:43.808826 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:43.808826 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:43.808826 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:43.808826 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:43.808826 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:43.808826 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:43.808826 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 22:26:43.808826 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 22:26:43.808826 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 22:26:43.808826 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 22:26:43.809833 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:43.809833 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:43.809833 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 22:26:43.809833 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 22:26:43.809833 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 22:26:43.809833 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 22:26:43.809833 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:43.809833 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:43.809833 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:43.809833 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:43.809833 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:43.809833 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:43.809833 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:43.809833 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:43.809833 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:43.809833 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:43.809833 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:43.809833 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:43.810788 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:43.810788 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:43.810788 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 22:26:43.810788 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 22:26:43.810788 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 22:26:43.810788 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 22:26:43.810788 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 22:26:43.810788 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 22:26:43.810788 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 22:26:43.810788 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:26:43.811862 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-3c421cd1-cfeb-41d0-8b51-c204194c25db None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 20 22:26:43.811862 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:43.811862 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:43.811862 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:43.811862 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:43.811862 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:43.811862 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:43.811862 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:43.811862 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:43.811862 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:43.811862 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:43.811862 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:43.811862 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 22:26:43.811862 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 22:26:43.811862 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 22:26:43.811862 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 22:26:43.812810 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:43.812810 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:43.812810 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 22:26:43.812810 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 22:26:43.812810 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 22:26:43.812810 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 22:26:43.812810 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:43.812810 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:43.812810 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:43.812810 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:43.812810 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:43.812810 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:43.812810 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:43.812810 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:43.812810 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:43.812810 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:43.812810 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:43.812810 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:43.813724 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:43.813724 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:43.813724 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 22:26:43.813724 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 22:26:43.813724 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 22:26:43.813724 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 22:26:43.813724 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 22:26:43.813724 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 22:26:43.813724 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 20 22:26:43.813724 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:26:43.815036 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 374/732] 10.222.0.81 () {56 vars in 1117 bytes} [Wed Jul 20 22:26:43 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:43.815336 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 359/733] 10.222.0.81 () {56 vars in 1096 bytes} [Wed Jul 20 22:26:43 2022] GET /identity/v3/users/swift => generated 83 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:43.816566 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bfe3c7-2d51-4614-98be-ea021293412e None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:43.816859 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bfe3c7-2d51-4614-98be-ea021293412e None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:43.817143 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bfe3c7-2d51-4614-98be-ea021293412e None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:43.817968 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f591415a-40c1-40de-ad8d-b6b25191772b None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:43.818424 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 375/734] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:26:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:26:43.818529 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f591415a-40c1-40de-ad8d-b6b25191772b None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:43.818916 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f591415a-40c1-40de-ad8d-b6b25191772b None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:43.819139 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bfe3c7-2d51-4614-98be-ea021293412e None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:43.824714 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 360/735] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:26:43 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:26:43.824714 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f591415a-40c1-40de-ad8d-b6b25191772b None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:43.965198 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f591415a-40c1-40de-ad8d-b6b25191772b None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:43.968883 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bfe3c7-2d51-4614-98be-ea021293412e None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:43.995196 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:43.998598 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c936938-ff00-4159-a4b9-6abfe5a63d40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:43.999314 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c936938-ff00-4159-a4b9-6abfe5a63d40 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:43.999624 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c936938-ff00-4159-a4b9-6abfe5a63d40 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:43.999894 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c936938-ff00-4159-a4b9-6abfe5a63d40 None admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:44.000722 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c936938-ff00-4159-a4b9-6abfe5a63d40 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:44.001036 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c936938-ff00-4159-a4b9-6abfe5a63d40 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:44.002451 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c936938-ff00-4159-a4b9-6abfe5a63d40 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:44.002736 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bb83cc-b57b-4d02-b2ea-ad2f4c9751a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:44.003969 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bb83cc-b57b-4d02-b2ea-ad2f4c9751a6 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:44.004257 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bb83cc-b57b-4d02-b2ea-ad2f4c9751a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:44.004542 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bb83cc-b57b-4d02-b2ea-ad2f4c9751a6 None admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:44.005372 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bb83cc-b57b-4d02-b2ea-ad2f4c9751a6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:44.005692 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bb83cc-b57b-4d02-b2ea-ad2f4c9751a6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:44.007265 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bb83cc-b57b-4d02-b2ea-ad2f4c9751a6 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:44.020502 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 361/736] 10.222.0.81 () {56 vars in 1135 bytes} [Wed Jul 20 22:26:43 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:44.021730 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1046a8e-49be-4d2d-91e8-a90402559769 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:44.022019 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1046a8e-49be-4d2d-91e8-a90402559769 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:44.022299 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1046a8e-49be-4d2d-91e8-a90402559769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:44.028744 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 376/737] 10.222.0.81 () {56 vars in 1149 bytes} [Wed Jul 20 22:26:43 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:44.029990 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f74a43-0af4-47a8-94fa-8993c3daa9a1 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:44.030284 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f74a43-0af4-47a8-94fa-8993c3daa9a1 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:44.030567 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f74a43-0af4-47a8-94fa-8993c3daa9a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:44.038090 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-c1046a8e-49be-4d2d-91e8-a90402559769 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:26:44.039525 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-c1046a8e-49be-4d2d-91e8-a90402559769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:26:44.048087 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-b1f74a43-0af4-47a8-94fa-8993c3daa9a1 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:26:44.048795 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:44.049249 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-b1f74a43-0af4-47a8-94fa-8993c3daa9a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:26:44.105410 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 362/738] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:26:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 22:26:44.106934 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f2e066-3323-4269-b033-8f4cf5333740 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:44.107277 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f2e066-3323-4269-b033-8f4cf5333740 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:44.107557 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f2e066-3323-4269-b033-8f4cf5333740 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:44.108816 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 363/739] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:26:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:26:44.109502 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f2e066-3323-4269-b033-8f4cf5333740 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:44.132150 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 377/740] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:26:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 22:26:44.133119 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f74a43-0af4-47a8-94fa-8993c3daa9a1 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:44.217048 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f2e066-3323-4269-b033-8f4cf5333740 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:44.234435 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f74a43-0af4-47a8-94fa-8993c3daa9a1 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:44.250924 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:44.251089 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009feb7b-b724-447e-9be4-2bbfd02743a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:44.251712 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009feb7b-b724-447e-9be4-2bbfd02743a1 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:44.251907 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009feb7b-b724-447e-9be4-2bbfd02743a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:44.252189 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009feb7b-b724-447e-9be4-2bbfd02743a1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:44.267295 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:44.270978 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddcbe7d-f153-408a-86dd-5dd4226bd0ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:44.271670 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddcbe7d-f153-408a-86dd-5dd4226bd0ec None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:44.271946 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddcbe7d-f153-408a-86dd-5dd4226bd0ec None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:44.272227 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddcbe7d-f153-408a-86dd-5dd4226bd0ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:44.320230 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009feb7b-b724-447e-9be4-2bbfd02743a1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:44.321712 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009feb7b-b724-447e-9be4-2bbfd02743a1 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:44.344918 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddcbe7d-f153-408a-86dd-5dd4226bd0ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:44.346340 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddcbe7d-f153-408a-86dd-5dd4226bd0ec None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:44.419067 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 364/741] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Jul 20 22:26:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 22:26:44.420354 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e043cf9-64c0-48e0-8dac-3f48b186a742 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:44.420645 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e043cf9-64c0-48e0-8dac-3f48b186a742 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:44.420920 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e043cf9-64c0-48e0-8dac-3f48b186a742 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:44.437007 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-6e043cf9-64c0-48e0-8dac-3f48b186a742 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:26:44.438210 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-6e043cf9-64c0-48e0-8dac-3f48b186a742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:26:44.444182 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 378/742] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Jul 20 22:26:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 22:26:44.444993 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddcbe7d-f153-408a-86dd-5dd4226bd0ec None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:44.463921 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:44.478159 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:44.504870 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 365/743] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:26:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 22:26:44.505677 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e043cf9-64c0-48e0-8dac-3f48b186a742 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:44.546714 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddcbe7d-f153-408a-86dd-5dd4226bd0ec None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:44.580511 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e5c552-0418-4676-9ce4-6f7c6858c7bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:44.581123 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e5c552-0418-4676-9ce4-6f7c6858c7bd None admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:44.581986 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e5c552-0418-4676-9ce4-6f7c6858c7bd None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:44.582268 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e5c552-0418-4676-9ce4-6f7c6858c7bd None admin] PATH_INFO: `/v3/services` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:44.582980 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e5c552-0418-4676-9ce4-6f7c6858c7bd None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:44.584454 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e5c552-0418-4676-9ce4-6f7c6858c7bd None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:44.606510 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e043cf9-64c0-48e0-8dac-3f48b186a742 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:44.639519 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedffca6-13cc-4359-bccd-576f07747ffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:44.640191 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedffca6-13cc-4359-bccd-576f07747ffd None admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:44.640480 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedffca6-13cc-4359-bccd-576f07747ffd None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:44.640762 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedffca6-13cc-4359-bccd-576f07747ffd None admin] PATH_INFO: `/v3/services` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:44.641480 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedffca6-13cc-4359-bccd-576f07747ffd None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:44.642844 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedffca6-13cc-4359-bccd-576f07747ffd None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:44.700915 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 379/744] 10.222.0.81 () {60 vars in 1141 bytes} [Wed Jul 20 22:26:44 2022] POST /identity/v3/services => generated 251 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:26:44.701807 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e5c552-0418-4676-9ce4-6f7c6858c7bd None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:44.736592 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 366/745] 10.222.0.81 () {60 vars in 1141 bytes} [Wed Jul 20 22:26:44 2022] POST /identity/v3/services => generated 240 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:26:44.738375 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedffca6-13cc-4359-bccd-576f07747ffd None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:44.798845 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:44.804481 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:44.839566 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e5c552-0418-4676-9ce4-6f7c6858c7bd None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:44.904858 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e701d76e-a85c-42cd-b991-e1db45127cad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:44.905931 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e701d76e-a85c-42cd-b991-e1db45127cad None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:44.906663 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e701d76e-a85c-42cd-b991-e1db45127cad None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:44.907094 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e701d76e-a85c-42cd-b991-e1db45127cad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:44.917081 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedffca6-13cc-4359-bccd-576f07747ffd None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:44.920395 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e701d76e-a85c-42cd-b991-e1db45127cad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:44.921877 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e701d76e-a85c-42cd-b991-e1db45127cad None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:44.946155 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-e701d76e-a85c-42cd-b991-e1db45127cad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:26:44.946155 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:44.946155 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:44.946155 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:44.946155 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:44.946155 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:44.946155 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:44.946155 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:44.946155 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:44.946155 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:44.946155 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:44.946155 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:44.946155 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 22:26:44.946155 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 22:26:44.946155 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 22:26:44.947134 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 22:26:44.947134 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:44.947134 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:44.947134 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:44.947134 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:44.947134 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:44.947134 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:44.947134 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:44.947134 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:44.947134 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:44.947134 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:44.947134 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:44.947134 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:44.947134 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:44.947134 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:44.947134 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:44.947134 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:44.948038 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 22:26:44.948038 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 22:26:44.948038 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:26:44.948038 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:26:44.948798 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 380/746] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Jul 20 22:26:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:44.949290 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e701d76e-a85c-42cd-b991-e1db45127cad None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:44.960036 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571c1c80-3861-4099-85a4-37de6d295881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:44.961220 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571c1c80-3861-4099-85a4-37de6d295881 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:44.961715 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571c1c80-3861-4099-85a4-37de6d295881 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:44.963104 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571c1c80-3861-4099-85a4-37de6d295881 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:44.975815 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571c1c80-3861-4099-85a4-37de6d295881 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:44.977749 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571c1c80-3861-4099-85a4-37de6d295881 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:45.004119 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-571c1c80-3861-4099-85a4-37de6d295881 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:26:45.004119 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:45.004119 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:45.004119 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:45.004119 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:45.004119 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:45.004119 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:45.004119 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:45.004119 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:45.004119 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:45.004119 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:45.004119 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:45.004119 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 22:26:45.004119 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 22:26:45.004119 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 22:26:45.005069 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 22:26:45.005069 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:45.005069 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:45.005069 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:45.005069 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:45.005069 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:45.005069 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:45.005069 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:45.005069 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:45.005069 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:45.005069 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:45.005069 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:45.005069 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:45.005069 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:45.005069 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:45.005069 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:45.005069 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:45.006000 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 22:26:45.006000 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 22:26:45.006000 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:26:45.006000 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:26:45.008271 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 367/747] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Jul 20 22:26:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:45.009262 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571c1c80-3861-4099-85a4-37de6d295881 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:45.101220 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e701d76e-a85c-42cd-b991-e1db45127cad None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:45.104886 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:45.124120 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571c1c80-3861-4099-85a4-37de6d295881 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:45.130428 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:45.149803 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb19ede7-41ba-4a6d-8bb0-c8b36096e2ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:45.151180 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb19ede7-41ba-4a6d-8bb0-c8b36096e2ee None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:45.151960 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb19ede7-41ba-4a6d-8bb0-c8b36096e2ee None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:45.152413 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb19ede7-41ba-4a6d-8bb0-c8b36096e2ee None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:45.161313 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bf09de-1bec-4243-b586-f0ccdddfcadc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:45.162224 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bf09de-1bec-4243-b586-f0ccdddfcadc None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:45.162456 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bf09de-1bec-4243-b586-f0ccdddfcadc None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:45.162734 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bf09de-1bec-4243-b586-f0ccdddfcadc None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:45.163891 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bf09de-1bec-4243-b586-f0ccdddfcadc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:45.164521 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bf09de-1bec-4243-b586-f0ccdddfcadc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:45.165802 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bf09de-1bec-4243-b586-f0ccdddfcadc None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:45.167188 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb19ede7-41ba-4a6d-8bb0-c8b36096e2ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 22:26:45.178845 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-eb19ede7-41ba-4a6d-8bb0-c8b36096e2ee None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 22:26:45.178845 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:45.178845 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:45.178845 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:45.178845 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:45.178845 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:45.178845 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:45.178845 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:45.178845 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:45.178845 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:45.178845 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:45.178845 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:45.178845 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 22:26:45.178845 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 22:26:45.178845 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 22:26:45.178845 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 22:26:45.179815 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 22:26:45.179815 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 22:26:45.179815 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 22:26:45.179815 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 22:26:45.179815 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:45.179815 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:45.179815 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:45.179815 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:45.179815 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:45.179815 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:45.179815 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:45.179815 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:45.179815 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:45.179815 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:45.179815 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:45.179815 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:45.180678 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:45.180678 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:45.180678 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:45.180678 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:45.180678 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 22:26:45.180678 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 22:26:45.180678 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 22:26:45.180678 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 22:26:45.180678 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 22:26:45.180678 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 22:26:45.180678 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 22:26:45.180678 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:26:45.183563 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 381/748] 10.222.0.81 () {56 vars in 1096 bytes} [Wed Jul 20 22:26:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:45.184564 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb19ede7-41ba-4a6d-8bb0-c8b36096e2ee None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:45.195001 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 368/749] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Jul 20 22:26:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:45.196068 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bf09de-1bec-4243-b586-f0ccdddfcadc None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:45.320420 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb19ede7-41ba-4a6d-8bb0-c8b36096e2ee None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:45.326661 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bf09de-1bec-4243-b586-f0ccdddfcadc None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:45.356433 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a448cc1-1ef5-4583-813e-1c4da3fd5c02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:45.357067 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a448cc1-1ef5-4583-813e-1c4da3fd5c02 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:45.357301 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a448cc1-1ef5-4583-813e-1c4da3fd5c02 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:45.357638 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a448cc1-1ef5-4583-813e-1c4da3fd5c02 None admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:45.358375 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a448cc1-1ef5-4583-813e-1c4da3fd5c02 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:45.358685 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a448cc1-1ef5-4583-813e-1c4da3fd5c02 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:45.360114 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a448cc1-1ef5-4583-813e-1c4da3fd5c02 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:45.374481 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3af3a37-52c0-46d3-9bb0-6779a1e6e525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:45.375506 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3af3a37-52c0-46d3-9bb0-6779a1e6e525 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:45.375988 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3af3a37-52c0-46d3-9bb0-6779a1e6e525 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:45.376488 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3af3a37-52c0-46d3-9bb0-6779a1e6e525 None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:45.377452 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3af3a37-52c0-46d3-9bb0-6779a1e6e525 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:45.377984 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3af3a37-52c0-46d3-9bb0-6779a1e6e525 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:45.378104 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 382/750] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Jul 20 22:26:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:45.378886 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a448cc1-1ef5-4583-813e-1c4da3fd5c02 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:45.380106 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3af3a37-52c0-46d3-9bb0-6779a1e6e525 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:45.396367 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 369/751] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Jul 20 22:26:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:45.397516 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3af3a37-52c0-46d3-9bb0-6779a1e6e525 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:45.404460 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:45.433796 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:45.505275 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3af3a37-52c0-46d3-9bb0-6779a1e6e525 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:45.509591 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a448cc1-1ef5-4583-813e-1c4da3fd5c02 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:45.546812 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d93224-c157-47dd-98a8-6f55c9931be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:45.548590 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d93224-c157-47dd-98a8-6f55c9931be7 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:45.549155 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d93224-c157-47dd-98a8-6f55c9931be7 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:45.549624 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d93224-c157-47dd-98a8-6f55c9931be7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:45.555496 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bd2235-7ce8-4e55-9b57-e802433e3d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:45.555496 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bd2235-7ce8-4e55-9b57-e802433e3d72 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:45.555496 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bd2235-7ce8-4e55-9b57-e802433e3d72 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:45.555496 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bd2235-7ce8-4e55-9b57-e802433e3d72 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:45.562379 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bd2235-7ce8-4e55-9b57-e802433e3d72 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:45.563328 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d93224-c157-47dd-98a8-6f55c9931be7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:45.564046 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bd2235-7ce8-4e55-9b57-e802433e3d72 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:45.565156 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d93224-c157-47dd-98a8-6f55c9931be7 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:45.579345 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-d8bd2235-7ce8-4e55-9b57-e802433e3d72 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 22:26:45.579345 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:45.579345 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:45.579345 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:45.579345 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:45.579345 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:45.579345 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:45.579345 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:45.579345 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:45.579345 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:45.579345 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:45.579345 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:45.579345 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 22:26:45.579345 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 22:26:45.579345 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 22:26:45.580209 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 22:26:45.580209 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:45.580209 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:45.580209 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:45.580209 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:45.580209 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:45.580209 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:45.580209 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:45.580209 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:45.580209 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:45.580209 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:45.580209 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:45.580209 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:45.580209 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:45.580209 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:45.580209 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:45.580209 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:45.581647 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 22:26:45.581647 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 22:26:45.581647 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 22:26:45.581647 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 22:26:45.581647 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 22:26:45.581647 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 22:26:45.581647 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 22:26:45.581647 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:26:45.581647 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 383/752] 10.222.0.81 () {56 vars in 1111 bytes} [Wed Jul 20 22:26:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:45.582270 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bd2235-7ce8-4e55-9b57-e802433e3d72 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:45.584719 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-b3d93224-c157-47dd-98a8-6f55c9931be7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 22:26:45.584719 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:45.584719 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:45.584719 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:45.584719 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:45.584719 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:45.584719 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:45.584719 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:45.584719 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:45.584719 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:45.584719 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:45.584719 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:45.584719 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 22:26:45.584719 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 22:26:45.584719 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 22:26:45.585736 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 22:26:45.585736 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:45.585736 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:45.585736 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:45.585736 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:45.585736 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:45.585736 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:45.585736 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:45.585736 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:45.585736 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:45.585736 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:45.585736 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:45.585736 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:45.585736 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:45.585736 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:45.585736 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:45.585736 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:45.586769 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 22:26:45.586769 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 22:26:45.586769 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 22:26:45.586769 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 22:26:45.586769 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 22:26:45.586769 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 22:26:45.586769 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 22:26:45.586769 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:26:45.590123 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 370/753] 10.222.0.81 () {56 vars in 1111 bytes} [Wed Jul 20 22:26:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:45.591180 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d93224-c157-47dd-98a8-6f55c9931be7 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:45.667736 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:45.704871 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d93224-c157-47dd-98a8-6f55c9931be7 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:45.710672 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:45.714519 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bd2235-7ce8-4e55-9b57-e802433e3d72 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:45.742308 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e48f5e0-dc66-4ae8-90d9-10927faa99eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:45.743132 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e48f5e0-dc66-4ae8-90d9-10927faa99eb None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:45.743492 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e48f5e0-dc66-4ae8-90d9-10927faa99eb None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:45.743841 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e48f5e0-dc66-4ae8-90d9-10927faa99eb None admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:45.744719 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e48f5e0-dc66-4ae8-90d9-10927faa99eb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:45.745111 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e48f5e0-dc66-4ae8-90d9-10927faa99eb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:45.746637 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e48f5e0-dc66-4ae8-90d9-10927faa99eb None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:45.758144 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151652ce-b2f6-4013-9ee6-5329daecd9e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:45.759265 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151652ce-b2f6-4013-9ee6-5329daecd9e6 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:45.759374 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 371/754] 10.222.0.81 () {56 vars in 1148 bytes} [Wed Jul 20 22:26:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:45.759879 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151652ce-b2f6-4013-9ee6-5329daecd9e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:45.760322 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151652ce-b2f6-4013-9ee6-5329daecd9e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:45.760469 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e48f5e0-dc66-4ae8-90d9-10927faa99eb None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:45.774537 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151652ce-b2f6-4013-9ee6-5329daecd9e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:45.776672 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151652ce-b2f6-4013-9ee6-5329daecd9e6 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:45.801130 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-151652ce-b2f6-4013-9ee6-5329daecd9e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:26:45.801130 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:45.801130 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:45.801130 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:45.801130 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:45.801130 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:45.801130 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:45.801130 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:45.801130 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:45.801130 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:45.801130 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:45.801130 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:45.801130 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 22:26:45.801130 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 22:26:45.801130 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 22:26:45.807612 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 22:26:45.807612 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:45.807612 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:45.807612 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:45.807612 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:45.807612 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:45.807612 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:45.807612 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:45.807612 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:45.807612 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:45.807612 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:45.807612 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:45.807612 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:45.807612 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:45.807612 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:45.807612 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:45.807612 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:45.808538 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 22:26:45.808538 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 22:26:45.808538 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:26:45.808538 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:26:45.808538 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 384/755] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Jul 20 22:26:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:45.808538 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151652ce-b2f6-4013-9ee6-5329daecd9e6 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:45.887268 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e48f5e0-dc66-4ae8-90d9-10927faa99eb None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:45.914461 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:45.925724 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37ba235-c5fe-45b0-a985-d4c89b8d30f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:45.926506 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37ba235-c5fe-45b0-a985-d4c89b8d30f7 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:45.926715 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37ba235-c5fe-45b0-a985-d4c89b8d30f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:45.927067 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37ba235-c5fe-45b0-a985-d4c89b8d30f7 None admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:45.927988 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37ba235-c5fe-45b0-a985-d4c89b8d30f7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:45.928308 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37ba235-c5fe-45b0-a985-d4c89b8d30f7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:45.933563 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37ba235-c5fe-45b0-a985-d4c89b8d30f7 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:45.939663 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151652ce-b2f6-4013-9ee6-5329daecd9e6 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:45.945237 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 372/756] 10.222.0.81 () {56 vars in 1148 bytes} [Wed Jul 20 22:26:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:45.951048 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37ba235-c5fe-45b0-a985-d4c89b8d30f7 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:45.992317 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459e7d87-051d-4872-8edc-e71136738a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:45.993187 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459e7d87-051d-4872-8edc-e71136738a4e None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:45.993571 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459e7d87-051d-4872-8edc-e71136738a4e None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:45.993947 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459e7d87-051d-4872-8edc-e71136738a4e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:45.994891 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459e7d87-051d-4872-8edc-e71136738a4e None admin] RBAC: Adding query filter params (role.id=d89b6c92d35a401a8517f1d47f6be843, scope.project.id=385edf2e59e546d090ace36600e29032, user.id=70e97436a08543a180225561ef8bc817) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:45.995319 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459e7d87-051d-4872-8edc-e71136738a4e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:45.996950 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459e7d87-051d-4872-8edc-e71136738a4e None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:46.006617 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 385/757] 10.222.0.81 () {56 vars in 1374 bytes} [Wed Jul 20 22:26:45 2022] GET /identity/v3/role_assignments?user.id=70e97436a08543a180225561ef8bc817&scope.project.id=385edf2e59e546d090ace36600e29032&role.id=d89b6c92d35a401a8517f1d47f6be843 => generated 259 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:46.008478 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459e7d87-051d-4872-8edc-e71136738a4e None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:46.048538 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:46.122821 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37ba235-c5fe-45b0-a985-d4c89b8d30f7 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:46.201098 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5a6419-b9de-4224-b334-a4e4c22275f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:46.201781 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5a6419-b9de-4224-b334-a4e4c22275f2 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:46.202067 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5a6419-b9de-4224-b334-a4e4c22275f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:46.202347 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5a6419-b9de-4224-b334-a4e4c22275f2 None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:46.203107 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5a6419-b9de-4224-b334-a4e4c22275f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:46.203423 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5a6419-b9de-4224-b334-a4e4c22275f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:46.204801 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5a6419-b9de-4224-b334-a4e4c22275f2 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:46.217588 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459e7d87-051d-4872-8edc-e71136738a4e None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:46.224440 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 373/758] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Jul 20 22:26:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:46.227985 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5a6419-b9de-4224-b334-a4e4c22275f2 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:46.288356 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90b0920-c169-4a32-975e-4c03cb2bfbf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:46.289465 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90b0920-c169-4a32-975e-4c03cb2bfbf0 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:46.290800 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90b0920-c169-4a32-975e-4c03cb2bfbf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:46.292269 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90b0920-c169-4a32-975e-4c03cb2bfbf0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:46.293562 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90b0920-c169-4a32-975e-4c03cb2bfbf0 None admin] RBAC: Adding query filter params (role.id=9b4a83e2eca44a8da9f356bc40f8788a, scope.project.id=385edf2e59e546d090ace36600e29032, user.id=90915e1264324f16bc7bae7e46e0c97b) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:46.295937 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90b0920-c169-4a32-975e-4c03cb2bfbf0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:46.297576 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90b0920-c169-4a32-975e-4c03cb2bfbf0 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:46.314503 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 386/759] 10.222.0.81 () {56 vars in 1374 bytes} [Wed Jul 20 22:26:46 2022] GET /identity/v3/role_assignments?user.id=90915e1264324f16bc7bae7e46e0c97b&scope.project.id=385edf2e59e546d090ace36600e29032&role.id=9b4a83e2eca44a8da9f356bc40f8788a => generated 259 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:46.322155 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:46.391168 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5a6419-b9de-4224-b334-a4e4c22275f2 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:46.438496 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c7c387-f4ad-4f7e-b317-685e6c0fd848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:46.439708 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c7c387-f4ad-4f7e-b317-685e6c0fd848 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:46.440214 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c7c387-f4ad-4f7e-b317-685e6c0fd848 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:46.441365 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c7c387-f4ad-4f7e-b317-685e6c0fd848 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:46.455310 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c7c387-f4ad-4f7e-b317-685e6c0fd848 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:46.457202 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c7c387-f4ad-4f7e-b317-685e6c0fd848 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:46.477140 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-84c7c387-f4ad-4f7e-b317-685e6c0fd848 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 20 22:26:46.477140 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:46.477140 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:46.477140 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:46.477140 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:46.477140 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:46.477140 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:46.477140 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:46.477140 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:46.477140 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:46.477140 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:46.477140 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:46.477140 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 22:26:46.477140 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 22:26:46.477140 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 22:26:46.477140 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 22:26:46.478161 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:46.478161 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:46.478161 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 22:26:46.478161 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 22:26:46.478161 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 22:26:46.478161 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 22:26:46.478161 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:46.478161 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:46.478161 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:46.478161 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:46.478161 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:46.478161 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:46.478161 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:46.478161 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:46.478161 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:46.478161 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:46.478161 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:46.478161 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:46.479119 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:46.479119 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:46.479119 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 22:26:46.479119 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 22:26:46.479119 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 22:26:46.479119 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 22:26:46.479119 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 22:26:46.479119 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 22:26:46.479119 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 20 22:26:46.479119 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:26:46.481958 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 374/760] 10.222.0.81 () {56 vars in 1093 bytes} [Wed Jul 20 22:26:46 2022] GET /identity/v3/users/nova => generated 82 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:46.488808 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90b0920-c169-4a32-975e-4c03cb2bfbf0 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:46.597536 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:46.617707 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90b0920-c169-4a32-975e-4c03cb2bfbf0 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:46.653367 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d4689f-2c03-4c6d-ad86-4e678b9b60c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:46.654056 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d4689f-2c03-4c6d-ad86-4e678b9b60c4 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:46.654346 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d4689f-2c03-4c6d-ad86-4e678b9b60c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:46.654631 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d4689f-2c03-4c6d-ad86-4e678b9b60c4 None admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:46.655507 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d4689f-2c03-4c6d-ad86-4e678b9b60c4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:46.655821 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d4689f-2c03-4c6d-ad86-4e678b9b60c4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:46.657254 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d4689f-2c03-4c6d-ad86-4e678b9b60c4 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:46.676906 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 387/761] 10.222.0.81 () {56 vars in 1133 bytes} [Wed Jul 20 22:26:46 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 22:26:46.682311 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c7c387-f4ad-4f7e-b317-685e6c0fd848 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:46.855625 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c7c387-f4ad-4f7e-b317-685e6c0fd848 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:46.864150 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:46.909895 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e750b10-cb67-4e6f-9e84-0d21245b6fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:46.910724 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e750b10-cb67-4e6f-9e84-0d21245b6fa2 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:46.911035 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e750b10-cb67-4e6f-9e84-0d21245b6fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:46.911319 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e750b10-cb67-4e6f-9e84-0d21245b6fa2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:46.930876 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e750b10-cb67-4e6f-9e84-0d21245b6fa2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:46.932368 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e750b10-cb67-4e6f-9e84-0d21245b6fa2 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:46.955278 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-2e750b10-cb67-4e6f-9e84-0d21245b6fa2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:26:46.955278 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:46.955278 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:46.955278 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:46.955278 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:46.955278 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:46.955278 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:46.955278 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:46.955278 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:46.955278 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:46.955278 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:46.955278 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:46.955278 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 22:26:46.955278 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 22:26:46.955278 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 22:26:46.956295 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 22:26:46.956295 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:46.956295 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:46.956295 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:46.956295 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:46.956295 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:46.956295 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:46.956295 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:46.956295 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:46.956295 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:46.956295 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:46.956295 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:46.956295 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:46.956295 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:46.956295 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:46.956295 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:46.956295 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:46.957318 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 22:26:46.957318 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 22:26:46.957318 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:26:46.957318 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:26:46.961053 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 375/762] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Jul 20 22:26:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:46.966730 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d4689f-2c03-4c6d-ad86-4e678b9b60c4 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:46.979892 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d683f501-5a8a-4071-9d01-30bc2b93990e None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:46.980483 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d683f501-5a8a-4071-9d01-30bc2b93990e None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:46.980988 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d683f501-5a8a-4071-9d01-30bc2b93990e None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:46.982789 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 376/763] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:26:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 22:26:46.990220 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb87914-d5cd-4173-95e0-cf22fae20d03 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:46.990748 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb87914-d5cd-4173-95e0-cf22fae20d03 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:46.991305 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb87914-d5cd-4173-95e0-cf22fae20d03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:47.027385 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-9cb87914-d5cd-4173-95e0-cf22fae20d03 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:26:47.028929 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-9cb87914-d5cd-4173-95e0-cf22fae20d03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:26:47.115303 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d4689f-2c03-4c6d-ad86-4e678b9b60c4 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:47.130523 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 377/764] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:26:46 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 22:26:47.135072 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a80477-6ba7-42e0-b714-cf4276d7038b None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:47.135072 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a80477-6ba7-42e0-b714-cf4276d7038b None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:47.135072 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a80477-6ba7-42e0-b714-cf4276d7038b None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:47.138426 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 378/765] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:26:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:26:47.140112 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbff2636-7622-4538-aed2-a9b4083c53ae None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:47.140410 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbff2636-7622-4538-aed2-a9b4083c53ae None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:47.140701 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbff2636-7622-4538-aed2-a9b4083c53ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:47.160797 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-bbff2636-7622-4538-aed2-a9b4083c53ae None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:26:47.162296 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-bbff2636-7622-4538-aed2-a9b4083c53ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:26:47.177112 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce102db-619a-4801-b94e-bc00c40046c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:47.178168 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce102db-619a-4801-b94e-bc00c40046c7 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:47.178753 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce102db-619a-4801-b94e-bc00c40046c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:47.179304 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce102db-619a-4801-b94e-bc00c40046c7 None admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:47.180357 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce102db-619a-4801-b94e-bc00c40046c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:47.180870 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce102db-619a-4801-b94e-bc00c40046c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:47.182591 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce102db-619a-4801-b94e-bc00c40046c7 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:47.191106 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:47.199944 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 388/766] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Jul 20 22:26:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:47.201218 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce102db-619a-4801-b94e-bc00c40046c7 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:47.248976 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 379/767] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:26:47 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 22:26:47.250146 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbff2636-7622-4538-aed2-a9b4083c53ae None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:47.279958 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:47.356682 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce102db-619a-4801-b94e-bc00c40046c7 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:47.372087 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbff2636-7622-4538-aed2-a9b4083c53ae None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:47.404143 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4872b0b5-c22d-4efe-a3b3-43b0a31e058c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:47.404965 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4872b0b5-c22d-4efe-a3b3-43b0a31e058c None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:47.405624 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4872b0b5-c22d-4efe-a3b3-43b0a31e058c None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:47.405932 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4872b0b5-c22d-4efe-a3b3-43b0a31e058c None admin] PATH_INFO: `/v3/services/network` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:47.406750 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4872b0b5-c22d-4efe-a3b3-43b0a31e058c None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:47.408288 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4872b0b5-c22d-4efe-a3b3-43b0a31e058c None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:47.411797 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a0b045-a976-4e69-aebf-ee234e285006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:47.413461 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a0b045-a976-4e69-aebf-ee234e285006 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:47.413944 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a0b045-a976-4e69-aebf-ee234e285006 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:47.414416 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a0b045-a976-4e69-aebf-ee234e285006 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:47.421625 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a0b045-a976-4e69-aebf-ee234e285006 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:47.422349 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-4872b0b5-c22d-4efe-a3b3-43b0a31e058c None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 20 22:26:47.422349 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:47.422349 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 22:26:47.422349 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 22:26:47.422349 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 22:26:47.422349 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 22:26:47.422349 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 22:26:47.422349 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 22:26:47.422349 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 20 22:26:47.422349 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:26:47.422349 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 22:26:47.422349 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:26:47.422349 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:47.422349 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:47.422349 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:47.422349 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:47.422349 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:47.423536 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:47.423536 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:47.423536 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:47.423536 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:47.423536 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:47.423536 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:47.423536 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 22:26:47.423536 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 22:26:47.423536 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 22:26:47.423536 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 22:26:47.423536 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:47.423536 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:47.423536 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:47.423536 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:47.423536 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:47.423536 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:47.424532 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:47.424532 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:47.424532 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:47.424532 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:47.424532 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:47.424532 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:47.424532 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:47.424532 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:47.424532 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:47.424532 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:47.424532 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 22:26:47.424532 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 22:26:47.424532 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 20 22:26:47.424532 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:26:47.427287 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a0b045-a976-4e69-aebf-ee234e285006 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:47.429452 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 389/768] 10.222.0.81 () {56 vars in 1111 bytes} [Wed Jul 20 22:26:47 2022] GET /identity/v3/services/network => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:47.430657 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4872b0b5-c22d-4efe-a3b3-43b0a31e058c None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:47.443591 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-c8a0b045-a976-4e69-aebf-ee234e285006 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 22:26:47.443591 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:47.443591 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:47.443591 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:47.443591 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:47.443591 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:47.443591 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:47.443591 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:47.443591 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:47.443591 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:47.443591 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:47.443591 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:47.443591 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 22:26:47.443591 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 22:26:47.443591 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 22:26:47.444538 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 22:26:47.444538 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:47.444538 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:47.444538 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:47.444538 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:47.444538 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:47.444538 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:47.444538 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:47.444538 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:47.444538 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:47.444538 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:47.444538 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:47.444538 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:47.444538 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:47.444538 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:47.444538 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:47.444538 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:47.445427 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 22:26:47.445427 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 22:26:47.445427 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 22:26:47.445427 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 22:26:47.445427 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 22:26:47.445427 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 22:26:47.445427 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 22:26:47.445427 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:26:47.448070 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 380/769] 10.222.0.81 () {56 vars in 1111 bytes} [Wed Jul 20 22:26:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:47.449037 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a0b045-a976-4e69-aebf-ee234e285006 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:47.504236 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:47.547446 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4872b0b5-c22d-4efe-a3b3-43b0a31e058c None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:47.548737 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:47.578936 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a0b045-a976-4e69-aebf-ee234e285006 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:47.588310 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b1fca9-1553-49f0-8c2a-dc4c88bc924a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:47.589542 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b1fca9-1553-49f0-8c2a-dc4c88bc924a None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:47.590221 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b1fca9-1553-49f0-8c2a-dc4c88bc924a None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:47.591105 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b1fca9-1553-49f0-8c2a-dc4c88bc924a None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:47.592430 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b1fca9-1553-49f0-8c2a-dc4c88bc924a None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:47.594038 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b1fca9-1553-49f0-8c2a-dc4c88bc924a None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:47.608082 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-e9b1fca9-1553-49f0-8c2a-dc4c88bc924a None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 20 22:26:47.608082 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:47.608082 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 22:26:47.608082 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 22:26:47.608082 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 22:26:47.608082 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 22:26:47.608082 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 22:26:47.608082 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 22:26:47.608082 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 20 22:26:47.608082 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:26:47.608082 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 22:26:47.608082 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:26:47.608082 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:47.608082 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:47.608082 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:47.608082 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:47.609057 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:47.609057 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:47.609057 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:47.609057 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:47.609057 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:47.609057 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:47.609057 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:47.609057 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 22:26:47.609057 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 22:26:47.609057 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 22:26:47.609057 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 22:26:47.609057 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:47.609057 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:47.609057 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:47.609057 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:47.609057 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:47.609904 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:47.609904 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:47.609904 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:47.609904 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:47.609904 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:47.609904 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:47.609904 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:47.609904 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:47.609904 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:47.609904 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:47.609904 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:47.609904 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 22:26:47.609904 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 22:26:47.609904 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 20 22:26:47.609904 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:26:47.612848 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 390/770] 10.222.0.81 () {56 vars in 1129 bytes} [Wed Jul 20 22:26:47 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:47.613910 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b1fca9-1553-49f0-8c2a-dc4c88bc924a None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:47.631821 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157e9257-7052-40f5-9cc0-807067e66f6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:47.631821 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157e9257-7052-40f5-9cc0-807067e66f6d None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:47.631821 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157e9257-7052-40f5-9cc0-807067e66f6d None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:47.631821 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157e9257-7052-40f5-9cc0-807067e66f6d None admin] PATH_INFO: `/v3/services` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:47.632459 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157e9257-7052-40f5-9cc0-807067e66f6d None admin] RBAC: Adding query filter params (name=network) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:47.632459 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157e9257-7052-40f5-9cc0-807067e66f6d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:47.632459 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157e9257-7052-40f5-9cc0-807067e66f6d None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:47.637819 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 381/771] 10.222.0.81 () {56 vars in 1112 bytes} [Wed Jul 20 22:26:47 2022] GET /identity/v3/services?name=network => generated 124 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:47.637819 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157e9257-7052-40f5-9cc0-807067e66f6d None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:47.748494 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157e9257-7052-40f5-9cc0-807067e66f6d None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:47.748899 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b1fca9-1553-49f0-8c2a-dc4c88bc924a None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:47.777442 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:47.786111 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8508b166-4f49-41ee-865b-6764083e85ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:47.793706 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f17a29-2a16-4f61-9da9-0677f454668e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:47.793706 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f17a29-2a16-4f61-9da9-0677f454668e None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:47.793911 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8508b166-4f49-41ee-865b-6764083e85ca None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:47.794000 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f17a29-2a16-4f61-9da9-0677f454668e None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:47.794087 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8508b166-4f49-41ee-865b-6764083e85ca None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:47.794181 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f17a29-2a16-4f61-9da9-0677f454668e None admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:47.794275 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8508b166-4f49-41ee-865b-6764083e85ca None admin] PATH_INFO: `/v3/services` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:47.794359 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f17a29-2a16-4f61-9da9-0677f454668e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:47.794471 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8508b166-4f49-41ee-865b-6764083e85ca None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:47.794605 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f17a29-2a16-4f61-9da9-0677f454668e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:47.794708 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8508b166-4f49-41ee-865b-6764083e85ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:47.794708 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8508b166-4f49-41ee-865b-6764083e85ca None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:47.794859 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f17a29-2a16-4f61-9da9-0677f454668e None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:47.796550 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 382/772] 10.222.0.81 () {56 vars in 1124 bytes} [Wed Jul 20 22:26:47 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:47.797498 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8508b166-4f49-41ee-865b-6764083e85ca None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:47.802212 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:47.805666 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 391/773] 10.222.0.81 () {56 vars in 1148 bytes} [Wed Jul 20 22:26:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:47.810668 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f17a29-2a16-4f61-9da9-0677f454668e None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:47.910169 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8508b166-4f49-41ee-865b-6764083e85ca None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:47.917760 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f17a29-2a16-4f61-9da9-0677f454668e None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:47.947248 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c909f2-37c6-4f04-a8c4-c4144befeb63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:47.948074 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c909f2-37c6-4f04-a8c4-c4144befeb63 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:47.948323 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c909f2-37c6-4f04-a8c4-c4144befeb63 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:47.948610 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c909f2-37c6-4f04-a8c4-c4144befeb63 None admin] PATH_INFO: `/v3/services` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:47.949375 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c909f2-37c6-4f04-a8c4-c4144befeb63 None admin] RBAC: Adding query filter params (type=network) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:47.949631 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c909f2-37c6-4f04-a8c4-c4144befeb63 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:47.951167 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c909f2-37c6-4f04-a8c4-c4144befeb63 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:47.961489 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 383/774] 10.222.0.81 () {56 vars in 1112 bytes} [Wed Jul 20 22:26:47 2022] GET /identity/v3/services?type=network => generated 350 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:47.961489 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c909f2-37c6-4f04-a8c4-c4144befeb63 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:47.964044 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c2bf39-1781-4328-9f58-8e232c6cc429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:47.964930 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c2bf39-1781-4328-9f58-8e232c6cc429 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:47.965391 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c2bf39-1781-4328-9f58-8e232c6cc429 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:47.965818 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c2bf39-1781-4328-9f58-8e232c6cc429 None admin] PATH_INFO: `/v3/services` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:47.966676 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c2bf39-1781-4328-9f58-8e232c6cc429 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:47.967178 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c2bf39-1781-4328-9f58-8e232c6cc429 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:47.968723 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c2bf39-1781-4328-9f58-8e232c6cc429 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:47.976495 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 392/775] 10.222.0.81 () {56 vars in 1124 bytes} [Wed Jul 20 22:26:47 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:47.977614 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c2bf39-1781-4328-9f58-8e232c6cc429 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:48.036067 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:48.049291 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:48.078590 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c909f2-37c6-4f04-a8c4-c4144befeb63 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:48.107269 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c2bf39-1781-4328-9f58-8e232c6cc429 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:48.124820 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ffc38b-d695-4b01-9439-cecbbc3d97a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:48.126200 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ffc38b-d695-4b01-9439-cecbbc3d97a6 None admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:48.127071 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ffc38b-d695-4b01-9439-cecbbc3d97a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:48.127503 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ffc38b-d695-4b01-9439-cecbbc3d97a6 None admin] PATH_INFO: `/v3/projects/385edf2e59e546d090ace36600e29032/users/7829324db2574d38b18c749d0559177f/roles/d89b6c92d35a401a8517f1d47f6be843` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:48.130768 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ffc38b-d695-4b01-9439-cecbbc3d97a6 None admin] RBAC: Authorizing `identity:create_grant(project_id=385edf2e59e546d090ace36600e29032, user_id=7829324db2574d38b18c749d0559177f, role_id=d89b6c92d35a401a8517f1d47f6be843)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:48.132706 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ffc38b-d695-4b01-9439-cecbbc3d97a6 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:48.145269 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538fe2a9-13c1-4b2d-8ff8-71e462123adb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:48.146032 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538fe2a9-13c1-4b2d-8ff8-71e462123adb None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:48.146445 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538fe2a9-13c1-4b2d-8ff8-71e462123adb None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:48.154370 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538fe2a9-13c1-4b2d-8ff8-71e462123adb None admin] PATH_INFO: `/v3/endpoints` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:48.154370 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538fe2a9-13c1-4b2d-8ff8-71e462123adb None admin] RBAC: Adding query filter params (interface=public, service_id=efc7ea2472aa4048a4a740869f7493c9, region_id=RegionOne) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:48.154370 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538fe2a9-13c1-4b2d-8ff8-71e462123adb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:48.154370 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538fe2a9-13c1-4b2d-8ff8-71e462123adb None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:48.157652 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 393/776] 10.222.0.81 () {56 vars in 1251 bytes} [Wed Jul 20 22:26:47 2022] GET /identity/v3/endpoints?service_id=efc7ea2472aa4048a4a740869f7493c9&interface=public®ion_id=RegionOne => generated 194 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:48.158716 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538fe2a9-13c1-4b2d-8ff8-71e462123adb None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:48.227310 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 384/777] 10.222.0.81 () {58 vars in 1439 bytes} [Wed Jul 20 22:26:47 2022] PUT /identity/v3/projects/385edf2e59e546d090ace36600e29032/users/7829324db2574d38b18c749d0559177f/roles/d89b6c92d35a401a8517f1d47f6be843 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:26:48.228330 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ffc38b-d695-4b01-9439-cecbbc3d97a6 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:48.296371 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538fe2a9-13c1-4b2d-8ff8-71e462123adb None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:48.313801 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:48.338786 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8536b8b-f8d1-4e54-ba48-1af565598f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:48.339744 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8536b8b-f8d1-4e54-ba48-1af565598f69 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:48.340139 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8536b8b-f8d1-4e54-ba48-1af565598f69 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:48.340505 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8536b8b-f8d1-4e54-ba48-1af565598f69 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:48.341452 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8536b8b-f8d1-4e54-ba48-1af565598f69 None admin] RBAC: Adding query filter params (interface=public, service_id=8642c81ce7834cc79e2313a5d9839b3e, region_id=RegionOne) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:48.341884 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8536b8b-f8d1-4e54-ba48-1af565598f69 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:48.343604 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8536b8b-f8d1-4e54-ba48-1af565598f69 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:48.349925 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 394/778] 10.222.0.81 () {56 vars in 1251 bytes} [Wed Jul 20 22:26:48 2022] GET /identity/v3/endpoints?service_id=8642c81ce7834cc79e2313a5d9839b3e&interface=public®ion_id=RegionOne => generated 194 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:48.351408 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33c4ff3-cc60-412b-a641-f8d1eadc10b7 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:48.351801 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33c4ff3-cc60-412b-a641-f8d1eadc10b7 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:48.352184 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33c4ff3-cc60-412b-a641-f8d1eadc10b7 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:48.353839 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 395/779] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:26:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:26:48.362377 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1057496-0238-4752-bd28-e4cfeda5a47f None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:48.362675 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1057496-0238-4752-bd28-e4cfeda5a47f None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:48.362965 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1057496-0238-4752-bd28-e4cfeda5a47f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:48.384612 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-d1057496-0238-4752-bd28-e4cfeda5a47f None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:26:48.386030 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-d1057496-0238-4752-bd28-e4cfeda5a47f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:26:48.412275 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ffc38b-d695-4b01-9439-cecbbc3d97a6 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:48.446576 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:48.454285 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81574173-1b3f-4336-b448-e66432e7418c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:48.455061 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81574173-1b3f-4336-b448-e66432e7418c None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:48.455563 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81574173-1b3f-4336-b448-e66432e7418c None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:48.455672 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81574173-1b3f-4336-b448-e66432e7418c None admin] PATH_INFO: `/v3/services` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:48.456882 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81574173-1b3f-4336-b448-e66432e7418c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:48.458279 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81574173-1b3f-4336-b448-e66432e7418c None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:48.463655 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 385/780] 10.222.0.81 () {56 vars in 1087 bytes} [Wed Jul 20 22:26:48 2022] GET /identity/v3/services => generated 772 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:48.464376 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81574173-1b3f-4336-b448-e66432e7418c None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:48.490472 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 396/781] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:26:48 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 22:26:48.515462 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1057496-0238-4752-bd28-e4cfeda5a47f None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:48.591441 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81574173-1b3f-4336-b448-e66432e7418c None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:48.623857 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69724fa1-bb5f-4752-8945-314e1b4552a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:48.625057 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69724fa1-bb5f-4752-8945-314e1b4552a1 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:48.625412 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69724fa1-bb5f-4752-8945-314e1b4552a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:48.625638 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69724fa1-bb5f-4752-8945-314e1b4552a1 None admin] PATH_INFO: `/v3/services` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:48.626347 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69724fa1-bb5f-4752-8945-314e1b4552a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:48.628325 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69724fa1-bb5f-4752-8945-314e1b4552a1 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:48.639262 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 386/782] 10.222.0.81 () {56 vars in 1087 bytes} [Wed Jul 20 22:26:48 2022] GET /identity/v3/services => generated 772 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:48.677944 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3a3ead-3750-4b1a-9cc8-40e1f508bd16 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:48.678267 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3a3ead-3750-4b1a-9cc8-40e1f508bd16 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:48.678516 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3a3ead-3750-4b1a-9cc8-40e1f508bd16 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:48.679918 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 387/783] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:26:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:26:48.685453 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85223a1-7c5a-4319-9274-8de5f9906cfe None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:48.685742 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85223a1-7c5a-4319-9274-8de5f9906cfe None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:48.686027 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85223a1-7c5a-4319-9274-8de5f9906cfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:48.698281 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1057496-0238-4752-bd28-e4cfeda5a47f None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:48.706790 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-e85223a1-7c5a-4319-9274-8de5f9906cfe None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:26:48.708082 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-e85223a1-7c5a-4319-9274-8de5f9906cfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:26:48.712674 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:48.754323 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5787ead-2ee4-4b18-a550-e0d47a10afd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:48.755199 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5787ead-2ee4-4b18-a550-e0d47a10afd5 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:48.756348 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5787ead-2ee4-4b18-a550-e0d47a10afd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:48.756348 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5787ead-2ee4-4b18-a550-e0d47a10afd5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:48.769009 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5787ead-2ee4-4b18-a550-e0d47a10afd5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 22:26:48.774329 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:48.777680 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-d5787ead-2ee4-4b18-a550-e0d47a10afd5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 22:26:48.777680 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:48.777680 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:48.777680 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:48.777680 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:48.777680 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:48.777680 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:48.777680 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:48.777680 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:48.777680 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:48.777680 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:48.777680 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:48.777680 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 22:26:48.777680 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 22:26:48.777680 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 22:26:48.777680 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 22:26:48.778809 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 22:26:48.778809 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 22:26:48.778809 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 22:26:48.778809 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 22:26:48.778809 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:48.778809 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:48.778809 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:48.778809 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:48.778809 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:48.778809 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:48.778809 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:48.778809 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:48.778809 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:48.778809 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:48.778809 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:48.778809 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:48.779830 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:48.779830 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:48.779830 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:48.779830 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:48.779830 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 22:26:48.779830 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 22:26:48.779830 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 22:26:48.779830 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 22:26:48.779830 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 22:26:48.779830 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 22:26:48.779830 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 22:26:48.779830 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:26:48.781600 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 397/784] 10.222.0.81 () {56 vars in 1096 bytes} [Wed Jul 20 22:26:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 22:26:48.788148 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5787ead-2ee4-4b18-a550-e0d47a10afd5 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:48.814339 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 388/785] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:26:48 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 22:26:48.847148 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85223a1-7c5a-4319-9274-8de5f9906cfe None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:48.921778 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5787ead-2ee4-4b18-a550-e0d47a10afd5 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:48.969941 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b645e7-5fb1-468b-a75c-dd81c28aedd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:48.970149 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85223a1-7c5a-4319-9274-8de5f9906cfe None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:48.971192 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b645e7-5fb1-468b-a75c-dd81c28aedd2 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:48.972220 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b645e7-5fb1-468b-a75c-dd81c28aedd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:48.972702 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b645e7-5fb1-468b-a75c-dd81c28aedd2 None admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:48.973714 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b645e7-5fb1-468b-a75c-dd81c28aedd2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:48.974226 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b645e7-5fb1-468b-a75c-dd81c28aedd2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:48.975838 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b645e7-5fb1-468b-a75c-dd81c28aedd2 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:48.994857 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 398/786] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Jul 20 22:26:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:49.010649 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b645e7-5fb1-468b-a75c-dd81c28aedd2 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:49.012925 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08968aef-c42c-48b6-833b-c6c4d81fbfaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:49.013752 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08968aef-c42c-48b6-833b-c6c4d81fbfaf None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:49.014033 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08968aef-c42c-48b6-833b-c6c4d81fbfaf None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:49.014307 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08968aef-c42c-48b6-833b-c6c4d81fbfaf None admin] PATH_INFO: `/v3/roles/service` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:49.028514 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08968aef-c42c-48b6-833b-c6c4d81fbfaf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 22:26:49.038549 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-08968aef-c42c-48b6-833b-c6c4d81fbfaf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 22:26:49.038549 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:49.038549 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:49.038549 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:49.038549 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:49.038549 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:49.038549 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:49.038549 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:49.038549 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:49.038549 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:49.038549 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:49.038549 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:49.038549 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 22:26:49.038549 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 22:26:49.038549 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 22:26:49.038549 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 22:26:49.039530 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 22:26:49.039530 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 22:26:49.039530 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 22:26:49.039530 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 22:26:49.039530 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:49.039530 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:49.039530 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:49.039530 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:49.039530 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:49.039530 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:49.039530 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:49.039530 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:49.039530 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:49.039530 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:49.039530 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:49.039530 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:49.040377 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:49.040377 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:49.040377 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:49.040377 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:49.040377 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 22:26:49.040377 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 22:26:49.040377 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 22:26:49.040377 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 22:26:49.040377 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 22:26:49.040377 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 22:26:49.040377 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 22:26:49.040377 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:26:49.040377 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 389/787] 10.222.0.81 () {56 vars in 1102 bytes} [Wed Jul 20 22:26:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 22:26:49.045467 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08968aef-c42c-48b6-833b-c6c4d81fbfaf None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:49.072943 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:49.130146 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:49.196572 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08968aef-c42c-48b6-833b-c6c4d81fbfaf None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:49.210704 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b645e7-5fb1-468b-a75c-dd81c28aedd2 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:49.233756 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232b2499-9065-4ebb-9ea0-75ec99938b0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:49.234362 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232b2499-9065-4ebb-9ea0-75ec99938b0e None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:49.234650 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232b2499-9065-4ebb-9ea0-75ec99938b0e None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:49.234931 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232b2499-9065-4ebb-9ea0-75ec99938b0e None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:49.235720 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232b2499-9065-4ebb-9ea0-75ec99938b0e None admin] RBAC: Adding query filter params (name=service) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:49.236032 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232b2499-9065-4ebb-9ea0-75ec99938b0e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:49.238039 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232b2499-9065-4ebb-9ea0-75ec99938b0e None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:49.247338 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 390/788] 10.222.0.81 () {56 vars in 1103 bytes} [Wed Jul 20 22:26:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 22:26:49.252381 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232b2499-9065-4ebb-9ea0-75ec99938b0e None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:49.289258 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe6deb0-12ee-46b0-8f33-8e54073f68bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:49.290075 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe6deb0-12ee-46b0-8f33-8e54073f68bb None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:49.290359 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe6deb0-12ee-46b0-8f33-8e54073f68bb None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:49.290637 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe6deb0-12ee-46b0-8f33-8e54073f68bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:49.308494 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe6deb0-12ee-46b0-8f33-8e54073f68bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:49.310227 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe6deb0-12ee-46b0-8f33-8e54073f68bb None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:49.331329 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-1fe6deb0-12ee-46b0-8f33-8e54073f68bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:26:49.331329 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:49.331329 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:49.331329 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:49.331329 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:49.331329 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:49.331329 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:49.331329 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:49.331329 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:49.331329 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:49.331329 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:49.331329 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:49.331329 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 22:26:49.331329 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 22:26:49.331329 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 22:26:49.332239 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 22:26:49.332239 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:49.332239 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:49.332239 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:49.332239 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:49.332239 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:49.332239 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:49.332239 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:49.332239 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:49.332239 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:49.332239 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:49.332239 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:49.332239 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:49.332239 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:49.332239 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:49.332239 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:49.332239 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:49.333250 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 22:26:49.333250 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 22:26:49.333250 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:26:49.333250 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:26:49.333250 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 399/789] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Jul 20 22:26:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 22:26:49.344001 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe6deb0-12ee-46b0-8f33-8e54073f68bb None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:49.403446 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232b2499-9065-4ebb-9ea0-75ec99938b0e None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:49.412571 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:49.444438 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d02ce1-1bc2-4904-8dff-eaf51eb6da76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:49.445304 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d02ce1-1bc2-4904-8dff-eaf51eb6da76 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:49.445591 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d02ce1-1bc2-4904-8dff-eaf51eb6da76 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:49.445865 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d02ce1-1bc2-4904-8dff-eaf51eb6da76 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:49.466590 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d02ce1-1bc2-4904-8dff-eaf51eb6da76 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:49.468090 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d02ce1-1bc2-4904-8dff-eaf51eb6da76 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:49.503601 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-70d02ce1-1bc2-4904-8dff-eaf51eb6da76 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:26:49.503601 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:49.503601 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:49.503601 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:49.503601 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:49.503601 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:49.503601 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:49.503601 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:49.503601 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:49.503601 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:49.503601 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:49.503601 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:49.503601 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 22:26:49.503601 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 22:26:49.503601 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 22:26:49.504493 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 22:26:49.504493 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:49.504493 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:49.504493 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:49.504493 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:49.504493 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:49.504493 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:49.504493 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:49.504493 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:49.504493 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:49.504493 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:49.504493 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:49.504493 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:49.504493 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:49.504493 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:49.504493 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:49.504493 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:49.505375 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 22:26:49.505375 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 22:26:49.505375 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:26:49.505375 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:26:49.506105 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 391/790] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Jul 20 22:26:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:49.511480 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d02ce1-1bc2-4904-8dff-eaf51eb6da76 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:49.519458 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:49.532356 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe6deb0-12ee-46b0-8f33-8e54073f68bb None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:49.568298 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2b3df6-b168-48f5-9fdc-e181f64c6df3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:49.569318 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2b3df6-b168-48f5-9fdc-e181f64c6df3 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:49.569802 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2b3df6-b168-48f5-9fdc-e181f64c6df3 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:49.570300 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2b3df6-b168-48f5-9fdc-e181f64c6df3 None admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:49.571246 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2b3df6-b168-48f5-9fdc-e181f64c6df3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:49.571752 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2b3df6-b168-48f5-9fdc-e181f64c6df3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:49.573304 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2b3df6-b168-48f5-9fdc-e181f64c6df3 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:49.590223 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 400/791] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Jul 20 22:26:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:49.591770 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2b3df6-b168-48f5-9fdc-e181f64c6df3 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:49.622213 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d02ce1-1bc2-4904-8dff-eaf51eb6da76 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:49.661519 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5325dcd8-9696-4315-84d9-31ad5253b605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:49.662208 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5325dcd8-9696-4315-84d9-31ad5253b605 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:49.662496 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5325dcd8-9696-4315-84d9-31ad5253b605 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:49.663059 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5325dcd8-9696-4315-84d9-31ad5253b605 None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:49.663801 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5325dcd8-9696-4315-84d9-31ad5253b605 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:49.664114 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5325dcd8-9696-4315-84d9-31ad5253b605 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:49.665661 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5325dcd8-9696-4315-84d9-31ad5253b605 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:49.678884 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 392/792] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Jul 20 22:26:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 22:26:49.684331 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5325dcd8-9696-4315-84d9-31ad5253b605 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:49.707576 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:49.777551 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2b3df6-b168-48f5-9fdc-e181f64c6df3 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:49.791760 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5325dcd8-9696-4315-84d9-31ad5253b605 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:49.828473 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7db623-f057-4483-8451-0e50f71416e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:49.829501 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7db623-f057-4483-8451-0e50f71416e9 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:49.829787 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7db623-f057-4483-8451-0e50f71416e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:49.830164 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7db623-f057-4483-8451-0e50f71416e9 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:49.838387 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7db623-f057-4483-8451-0e50f71416e9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:49.839971 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7db623-f057-4483-8451-0e50f71416e9 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:49.856469 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-ed7db623-f057-4483-8451-0e50f71416e9 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 22:26:49.856469 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:49.856469 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:49.856469 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:49.856469 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:49.856469 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:49.856469 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:49.856469 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:49.856469 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:49.856469 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:49.856469 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:49.856469 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:49.856469 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 22:26:49.856469 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 22:26:49.856469 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 22:26:49.856469 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 22:26:49.857402 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:49.857402 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:49.857402 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 22:26:49.857402 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 22:26:49.857402 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 22:26:49.857402 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 22:26:49.857402 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:49.857402 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:49.857402 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:49.857402 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:49.857402 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:49.857402 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:49.857402 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:49.857402 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:49.857402 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:49.857402 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:49.857402 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:49.857402 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:49.858731 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:49.858731 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:49.858731 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 22:26:49.858731 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 22:26:49.858731 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 22:26:49.858731 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 22:26:49.858731 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 22:26:49.858731 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 22:26:49.858731 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 22:26:49.858731 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:26:49.858731 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 393/793] 10.222.0.81 () {56 vars in 1117 bytes} [Wed Jul 20 22:26:49 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 22:26:49.863290 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25af9de-af84-4976-9034-d0cbae1fd671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:49.864108 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7db623-f057-4483-8451-0e50f71416e9 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:49.864240 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25af9de-af84-4976-9034-d0cbae1fd671 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:49.864399 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25af9de-af84-4976-9034-d0cbae1fd671 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:49.864764 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25af9de-af84-4976-9034-d0cbae1fd671 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:49.880103 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25af9de-af84-4976-9034-d0cbae1fd671 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:49.881545 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25af9de-af84-4976-9034-d0cbae1fd671 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:49.903472 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-e25af9de-af84-4976-9034-d0cbae1fd671 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 20 22:26:49.903472 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:49.903472 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:49.903472 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:49.903472 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:49.903472 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:49.903472 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:49.903472 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:49.903472 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:49.903472 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:49.903472 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:49.903472 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:49.903472 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 22:26:49.903472 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 22:26:49.903472 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 22:26:49.903472 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 22:26:49.904404 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:49.904404 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:49.904404 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 22:26:49.904404 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 22:26:49.904404 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 22:26:49.904404 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 22:26:49.904404 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:49.904404 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:49.904404 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:49.904404 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:49.904404 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:49.904404 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:49.904404 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:49.904404 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:49.904404 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:49.904404 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:49.904404 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:49.904404 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:49.905334 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:49.905334 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:49.905334 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 22:26:49.905334 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 22:26:49.905334 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 22:26:49.905334 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 22:26:49.905334 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 22:26:49.905334 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 22:26:49.905334 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 20 22:26:49.905334 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:26:49.911450 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 401/794] 10.222.0.81 () {56 vars in 1096 bytes} [Wed Jul 20 22:26:49 2022] GET /identity/v3/users/swift => generated 83 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 22:26:49.917431 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25af9de-af84-4976-9034-d0cbae1fd671 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:49.923894 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:49.948719 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:49.969193 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7db623-f057-4483-8451-0e50f71416e9 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:50.007604 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0697570-5d31-4542-a5e2-d98af3c210a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:50.008296 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0697570-5d31-4542-a5e2-d98af3c210a1 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:50.008582 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0697570-5d31-4542-a5e2-d98af3c210a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:50.008859 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0697570-5d31-4542-a5e2-d98af3c210a1 None admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:50.009696 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0697570-5d31-4542-a5e2-d98af3c210a1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:50.010008 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0697570-5d31-4542-a5e2-d98af3c210a1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:50.011468 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0697570-5d31-4542-a5e2-d98af3c210a1 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:50.038048 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 394/795] 10.222.0.81 () {56 vars in 1149 bytes} [Wed Jul 20 22:26:49 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 22:26:50.043419 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0697570-5d31-4542-a5e2-d98af3c210a1 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:50.105064 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25af9de-af84-4976-9034-d0cbae1fd671 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:50.142191 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8f5ef8-847d-4943-927e-acfb23fa8cb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:50.142880 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8f5ef8-847d-4943-927e-acfb23fa8cb1 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:50.143255 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8f5ef8-847d-4943-927e-acfb23fa8cb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:50.143546 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8f5ef8-847d-4943-927e-acfb23fa8cb1 None admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:50.144393 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8f5ef8-847d-4943-927e-acfb23fa8cb1 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:50.144710 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8f5ef8-847d-4943-927e-acfb23fa8cb1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:50.146256 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8f5ef8-847d-4943-927e-acfb23fa8cb1 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:50.152506 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0697570-5d31-4542-a5e2-d98af3c210a1 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:50.191726 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8873900c-4cec-41fe-9baa-5b5d3947eda1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:50.192604 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8873900c-4cec-41fe-9baa-5b5d3947eda1 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:50.192893 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8873900c-4cec-41fe-9baa-5b5d3947eda1 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:50.193172 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8873900c-4cec-41fe-9baa-5b5d3947eda1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:50.197373 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 402/796] 10.222.0.81 () {56 vars in 1135 bytes} [Wed Jul 20 22:26:49 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:50.200660 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:50.202927 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8f5ef8-847d-4943-927e-acfb23fa8cb1 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:50.206546 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8873900c-4cec-41fe-9baa-5b5d3947eda1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:50.208257 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8873900c-4cec-41fe-9baa-5b5d3947eda1 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:50.227576 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-8873900c-4cec-41fe-9baa-5b5d3947eda1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:26:50.227576 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:50.227576 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:50.227576 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:50.227576 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:50.227576 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:50.227576 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:50.227576 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:50.227576 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:50.227576 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:50.227576 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:50.227576 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:50.227576 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 22:26:50.227576 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 22:26:50.227576 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 22:26:50.228712 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 22:26:50.228712 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:50.228712 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:50.228712 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:50.228712 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:50.228712 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:50.228712 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:50.228712 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:50.228712 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:50.228712 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:50.228712 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:50.228712 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:50.228712 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:50.228712 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:50.228712 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:50.228712 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:50.228712 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:50.229807 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 22:26:50.229807 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 22:26:50.229807 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:26:50.229807 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:26:50.232726 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 395/797] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Jul 20 22:26:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 22:26:50.238713 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8873900c-4cec-41fe-9baa-5b5d3947eda1 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:50.344821 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8873900c-4cec-41fe-9baa-5b5d3947eda1 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:50.378937 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:50.381673 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5d75e6-0742-4411-9276-21302c545a12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:50.382518 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5d75e6-0742-4411-9276-21302c545a12 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:50.383004 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5d75e6-0742-4411-9276-21302c545a12 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:50.383451 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5d75e6-0742-4411-9276-21302c545a12 None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:50.384365 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5d75e6-0742-4411-9276-21302c545a12 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:50.384912 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5d75e6-0742-4411-9276-21302c545a12 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:50.386621 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5d75e6-0742-4411-9276-21302c545a12 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:50.399051 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 396/798] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Jul 20 22:26:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 22:26:50.404676 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5d75e6-0742-4411-9276-21302c545a12 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:50.457416 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:50.497186 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8f5ef8-847d-4943-927e-acfb23fa8cb1 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:50.511557 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5d75e6-0742-4411-9276-21302c545a12 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:50.550707 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79ec1fa-ff6d-4caa-93ab-c401eee5afb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:50.551809 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79ec1fa-ff6d-4caa-93ab-c401eee5afb5 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:50.552245 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79ec1fa-ff6d-4caa-93ab-c401eee5afb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:50.553189 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79ec1fa-ff6d-4caa-93ab-c401eee5afb5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:50.560896 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79ec1fa-ff6d-4caa-93ab-c401eee5afb5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:50.562510 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79ec1fa-ff6d-4caa-93ab-c401eee5afb5 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:50.577588 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8edf9fd-22a3-4f00-9c83-d69311ac9581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:50.578414 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8edf9fd-22a3-4f00-9c83-d69311ac9581 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:50.578701 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8edf9fd-22a3-4f00-9c83-d69311ac9581 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:50.578989 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8edf9fd-22a3-4f00-9c83-d69311ac9581 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:50.589403 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8edf9fd-22a3-4f00-9c83-d69311ac9581 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:50.590849 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8edf9fd-22a3-4f00-9c83-d69311ac9581 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:50.591580 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-e79ec1fa-ff6d-4caa-93ab-c401eee5afb5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 22:26:50.591580 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:50.591580 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:50.591580 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:50.591580 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:50.591580 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:50.591580 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:50.591580 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:50.591580 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:50.591580 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:50.591580 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:50.591580 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:50.591580 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 22:26:50.591580 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 22:26:50.591580 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 22:26:50.592612 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 22:26:50.592612 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:50.592612 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:50.592612 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:50.592612 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:50.592612 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:50.592612 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:50.592612 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:50.592612 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:50.592612 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:50.592612 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:50.592612 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:50.592612 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:50.592612 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:50.592612 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:50.592612 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:50.592612 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:50.593692 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 22:26:50.593692 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 22:26:50.593692 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 22:26:50.593692 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 22:26:50.593692 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 22:26:50.593692 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 22:26:50.593692 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 22:26:50.593692 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:26:50.595879 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 397/799] 10.222.0.81 () {56 vars in 1111 bytes} [Wed Jul 20 22:26:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:50.602291 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79ec1fa-ff6d-4caa-93ab-c401eee5afb5 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:50.611184 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-b8edf9fd-22a3-4f00-9c83-d69311ac9581 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:26:50.611184 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:50.611184 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:50.611184 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:50.611184 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:50.611184 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:50.611184 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:50.611184 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:50.611184 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:50.611184 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:50.611184 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:50.611184 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:50.611184 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 22:26:50.611184 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 22:26:50.611184 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 22:26:50.612808 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 22:26:50.612808 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:50.612808 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:50.612808 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:50.612808 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:50.612808 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:50.612808 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:50.612808 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:50.612808 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:50.612808 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:50.612808 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:50.612808 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:50.612808 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:50.612808 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:50.612808 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:50.612808 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:50.612808 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:50.613895 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 22:26:50.613895 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 22:26:50.613895 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:26:50.613895 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:26:50.615034 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 403/800] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Jul 20 22:26:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 412 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:50.628014 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8edf9fd-22a3-4f00-9c83-d69311ac9581 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:50.720336 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79ec1fa-ff6d-4caa-93ab-c401eee5afb5 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:50.732725 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:50.761172 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97be4d4-3893-4a2d-80b7-ea6f14fc87c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:50.762145 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97be4d4-3893-4a2d-80b7-ea6f14fc87c1 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:50.762608 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97be4d4-3893-4a2d-80b7-ea6f14fc87c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:50.763181 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97be4d4-3893-4a2d-80b7-ea6f14fc87c1 None admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:50.764163 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97be4d4-3893-4a2d-80b7-ea6f14fc87c1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:50.764690 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97be4d4-3893-4a2d-80b7-ea6f14fc87c1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:50.766312 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97be4d4-3893-4a2d-80b7-ea6f14fc87c1 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:50.773324 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8edf9fd-22a3-4f00-9c83-d69311ac9581 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:50.780196 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 398/801] 10.222.0.81 () {56 vars in 1148 bytes} [Wed Jul 20 22:26:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 22:26:50.787209 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e5261d-b24d-4189-8a39-3e4f33183c3a None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:50.788037 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e5261d-b24d-4189-8a39-3e4f33183c3a None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:50.788534 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e5261d-b24d-4189-8a39-3e4f33183c3a None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:50.791305 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 399/802] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:26:50 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:26:50.797926 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e5261d-b24d-4189-8a39-3e4f33183c3a None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:50.798052 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:50.820258 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede69e0b-e9f2-4b8f-b968-81e6b66a41e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:50.820950 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede69e0b-e9f2-4b8f-b968-81e6b66a41e2 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:50.821485 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede69e0b-e9f2-4b8f-b968-81e6b66a41e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:50.821777 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede69e0b-e9f2-4b8f-b968-81e6b66a41e2 None admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:50.822514 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede69e0b-e9f2-4b8f-b968-81e6b66a41e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:50.822842 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede69e0b-e9f2-4b8f-b968-81e6b66a41e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:50.824977 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede69e0b-e9f2-4b8f-b968-81e6b66a41e2 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:50.840231 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 404/803] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Jul 20 22:26:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:50.842088 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de35f80-d5d8-4616-92da-5db19ad7a535 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:50.842390 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de35f80-d5d8-4616-92da-5db19ad7a535 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:50.842675 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de35f80-d5d8-4616-92da-5db19ad7a535 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:50.864551 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-8de35f80-d5d8-4616-92da-5db19ad7a535 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:26:50.864551 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-8de35f80-d5d8-4616-92da-5db19ad7a535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:26:50.921879 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e5261d-b24d-4189-8a39-3e4f33183c3a None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:50.934175 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 405/804] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:26:50 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 22:26:50.935952 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de35f80-d5d8-4616-92da-5db19ad7a535 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:50.978833 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6141ffa8-46c7-448c-ae46-c5c3242e1fbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:50.980296 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6141ffa8-46c7-448c-ae46-c5c3242e1fbf None admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:50.980825 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6141ffa8-46c7-448c-ae46-c5c3242e1fbf None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:50.981313 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6141ffa8-46c7-448c-ae46-c5c3242e1fbf None admin] PATH_INFO: `/v3/projects/385edf2e59e546d090ace36600e29032/users/90915e1264324f16bc7bae7e46e0c97b/roles/9b4a83e2eca44a8da9f356bc40f8788a` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:50.997124 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6141ffa8-46c7-448c-ae46-c5c3242e1fbf None admin] RBAC: Authorizing `identity:create_grant(project_id=385edf2e59e546d090ace36600e29032, user_id=90915e1264324f16bc7bae7e46e0c97b, role_id=9b4a83e2eca44a8da9f356bc40f8788a)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:50.998901 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6141ffa8-46c7-448c-ae46-c5c3242e1fbf None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:51.061375 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de35f80-d5d8-4616-92da-5db19ad7a535 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:51.063066 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:51.072168 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 400/805] 10.222.0.81 () {58 vars in 1439 bytes} [Wed Jul 20 22:26:50 2022] PUT /identity/v3/projects/385edf2e59e546d090ace36600e29032/users/90915e1264324f16bc7bae7e46e0c97b/roles/9b4a83e2eca44a8da9f356bc40f8788a => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:26:51.078591 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6141ffa8-46c7-448c-ae46-c5c3242e1fbf None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:51.113232 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af484679-3e7e-413a-80f3-7e3b8584da3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:51.119775 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af484679-3e7e-413a-80f3-7e3b8584da3e None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:51.119775 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af484679-3e7e-413a-80f3-7e3b8584da3e None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:51.119775 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af484679-3e7e-413a-80f3-7e3b8584da3e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:51.120133 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:51.125632 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af484679-3e7e-413a-80f3-7e3b8584da3e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:51.127854 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af484679-3e7e-413a-80f3-7e3b8584da3e None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:51.148844 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-af484679-3e7e-413a-80f3-7e3b8584da3e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 22:26:51.148844 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:51.148844 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:51.148844 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:51.148844 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:51.148844 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:51.148844 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:51.148844 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:51.148844 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:51.148844 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:51.148844 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:51.148844 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:51.148844 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 22:26:51.148844 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 22:26:51.148844 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 22:26:51.149800 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 22:26:51.149800 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:51.149800 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:51.149800 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:51.149800 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:51.149800 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:51.149800 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:51.149800 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:51.149800 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:51.149800 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:51.149800 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:51.149800 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:51.149800 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:51.149800 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:51.149800 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:51.149800 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:51.149800 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:51.150715 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 22:26:51.150715 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 22:26:51.150715 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 22:26:51.150715 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 22:26:51.150715 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 22:26:51.150715 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 22:26:51.150715 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 22:26:51.150715 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:26:51.150715 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 406/806] 10.222.0.81 () {56 vars in 1111 bytes} [Wed Jul 20 22:26:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:51.156233 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af484679-3e7e-413a-80f3-7e3b8584da3e None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:51.280427 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6141ffa8-46c7-448c-ae46-c5c3242e1fbf None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:51.301584 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af484679-3e7e-413a-80f3-7e3b8584da3e None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:51.327401 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0195296-5237-44bc-b10d-8df1f16d698e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:51.328223 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0195296-5237-44bc-b10d-8df1f16d698e None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:51.328505 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0195296-5237-44bc-b10d-8df1f16d698e None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:51.328781 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0195296-5237-44bc-b10d-8df1f16d698e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:51.337775 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656af8dc-62cd-47f3-80ca-bfabba26d62d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:51.338435 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656af8dc-62cd-47f3-80ca-bfabba26d62d None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:51.338746 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656af8dc-62cd-47f3-80ca-bfabba26d62d None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:51.339043 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656af8dc-62cd-47f3-80ca-bfabba26d62d None admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:51.339535 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0195296-5237-44bc-b10d-8df1f16d698e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 22:26:51.339828 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656af8dc-62cd-47f3-80ca-bfabba26d62d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:51.340137 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656af8dc-62cd-47f3-80ca-bfabba26d62d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:51.341553 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656af8dc-62cd-47f3-80ca-bfabba26d62d None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:51.348372 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-d0195296-5237-44bc-b10d-8df1f16d698e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 22:26:51.348372 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:51.348372 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:51.348372 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:51.348372 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:51.348372 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:51.348372 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:51.348372 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:51.348372 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:51.348372 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:51.348372 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:51.348372 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:51.348372 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 22:26:51.348372 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 22:26:51.348372 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 22:26:51.348372 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 22:26:51.349270 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 22:26:51.349270 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 22:26:51.349270 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 22:26:51.349270 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 22:26:51.349270 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:51.349270 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:51.349270 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:51.349270 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:51.349270 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:51.349270 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:51.349270 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:51.349270 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:51.349270 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:51.349270 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:51.349270 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:51.349270 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:51.350196 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:51.350196 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:51.350196 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:51.350196 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:51.350196 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 22:26:51.350196 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 22:26:51.350196 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 22:26:51.350196 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 22:26:51.350196 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 22:26:51.350196 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 22:26:51.350196 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 22:26:51.350196 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:26:51.350196 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 401/807] 10.222.0.81 () {56 vars in 1096 bytes} [Wed Jul 20 22:26:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:51.350869 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5fa32c-15ab-4906-8aca-52bc2a37134f None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:51.351721 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5fa32c-15ab-4906-8aca-52bc2a37134f None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:51.351721 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5fa32c-15ab-4906-8aca-52bc2a37134f None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:51.353597 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 402/808] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:26:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:26:51.355002 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d65550-aad1-47d3-8eae-6ae8b713b816 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:51.355310 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d65550-aad1-47d3-8eae-6ae8b713b816 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:51.355584 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d65550-aad1-47d3-8eae-6ae8b713b816 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:51.363037 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 403/809] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:26:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:26:51.363037 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d65550-aad1-47d3-8eae-6ae8b713b816 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:51.363252 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 407/810] 10.222.0.81 () {56 vars in 1148 bytes} [Wed Jul 20 22:26:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:51.363552 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7148a90-e4e4-4efc-bce6-5c481149da98 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:51.363844 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7148a90-e4e4-4efc-bce6-5c481149da98 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:51.364118 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7148a90-e4e4-4efc-bce6-5c481149da98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:51.380540 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-e7148a90-e4e4-4efc-bce6-5c481149da98 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:26:51.381635 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-e7148a90-e4e4-4efc-bce6-5c481149da98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:26:51.400162 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:51.448667 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:51.463034 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 408/811] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:26:51 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 22:26:51.463034 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3c5293-5929-44e6-803d-40d21de1f9a6 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:51.463034 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3c5293-5929-44e6-803d-40d21de1f9a6 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:51.463034 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3c5293-5929-44e6-803d-40d21de1f9a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:51.477603 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d65550-aad1-47d3-8eae-6ae8b713b816 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:51.482036 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-dd3c5293-5929-44e6-803d-40d21de1f9a6 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:26:51.483514 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-dd3c5293-5929-44e6-803d-40d21de1f9a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:26:51.514657 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ed96fa-ccd9-43f0-8b6a-8d35e86c1c9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:51.515364 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ed96fa-ccd9-43f0-8b6a-8d35e86c1c9e None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:51.515648 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ed96fa-ccd9-43f0-8b6a-8d35e86c1c9e None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:51.515922 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ed96fa-ccd9-43f0-8b6a-8d35e86c1c9e None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:51.516654 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ed96fa-ccd9-43f0-8b6a-8d35e86c1c9e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:51.516960 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ed96fa-ccd9-43f0-8b6a-8d35e86c1c9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:51.519917 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ed96fa-ccd9-43f0-8b6a-8d35e86c1c9e None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:51.528671 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 404/812] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Jul 20 22:26:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:51.529442 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ed96fa-ccd9-43f0-8b6a-8d35e86c1c9e None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:51.559497 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 409/813] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:26:51 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 22:26:51.569036 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3c5293-5929-44e6-803d-40d21de1f9a6 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:51.635002 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ed96fa-ccd9-43f0-8b6a-8d35e86c1c9e None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:51.658268 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:51.672468 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62214205-3013-4d53-802a-997963b0d36d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:51.673369 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62214205-3013-4d53-802a-997963b0d36d None admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:51.673651 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62214205-3013-4d53-802a-997963b0d36d None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:51.673936 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62214205-3013-4d53-802a-997963b0d36d None admin] PATH_INFO: `/v3/projects/385edf2e59e546d090ace36600e29032/users/70e97436a08543a180225561ef8bc817/roles/d89b6c92d35a401a8517f1d47f6be843` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:51.676462 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62214205-3013-4d53-802a-997963b0d36d None admin] RBAC: Authorizing `identity:create_grant(project_id=385edf2e59e546d090ace36600e29032, user_id=70e97436a08543a180225561ef8bc817, role_id=d89b6c92d35a401a8517f1d47f6be843)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:51.677255 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3c5293-5929-44e6-803d-40d21de1f9a6 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:51.677880 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62214205-3013-4d53-802a-997963b0d36d None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:51.711536 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12349e15-92bc-457a-895b-11feb4cb450a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:51.712341 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12349e15-92bc-457a-895b-11feb4cb450a None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:51.712624 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12349e15-92bc-457a-895b-11feb4cb450a None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:51.712901 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12349e15-92bc-457a-895b-11feb4cb450a None admin] PATH_INFO: `/v3/services/network` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:51.713547 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12349e15-92bc-457a-895b-11feb4cb450a None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:51.714910 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12349e15-92bc-457a-895b-11feb4cb450a None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:51.726814 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-12349e15-92bc-457a-895b-11feb4cb450a None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 20 22:26:51.726814 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:51.726814 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 22:26:51.726814 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 22:26:51.726814 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 22:26:51.726814 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 22:26:51.726814 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 22:26:51.726814 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 22:26:51.726814 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 20 22:26:51.726814 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:26:51.726814 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 22:26:51.726814 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:26:51.726814 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:51.726814 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:51.726814 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:51.726814 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:51.726814 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:51.727982 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:51.727982 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:51.727982 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:51.727982 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:51.727982 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:51.727982 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:51.727982 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 22:26:51.727982 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 22:26:51.727982 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 22:26:51.727982 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 22:26:51.727982 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:51.727982 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:51.727982 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:51.727982 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:51.727982 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:51.727982 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:51.728837 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:51.728837 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:51.728837 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:51.728837 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:51.728837 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:51.728837 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:51.728837 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:51.728837 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:51.728837 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:51.728837 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:51.728837 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 22:26:51.728837 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 22:26:51.728837 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 20 22:26:51.728837 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:26:51.728837 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 410/814] 10.222.0.81 () {56 vars in 1111 bytes} [Wed Jul 20 22:26:51 2022] GET /identity/v3/services/network => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:51.730039 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12349e15-92bc-457a-895b-11feb4cb450a None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:51.755265 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 405/815] 10.222.0.81 () {58 vars in 1439 bytes} [Wed Jul 20 22:26:51 2022] PUT /identity/v3/projects/385edf2e59e546d090ace36600e29032/users/70e97436a08543a180225561ef8bc817/roles/d89b6c92d35a401a8517f1d47f6be843 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:26:51.755265 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62214205-3013-4d53-802a-997963b0d36d None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:51.755265 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:51.858009 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12349e15-92bc-457a-895b-11feb4cb450a None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:51.899635 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62214205-3013-4d53-802a-997963b0d36d None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:51.901194 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a68934-f2d0-4dc0-a701-7bed3553f7e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:51.902308 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a68934-f2d0-4dc0-a701-7bed3553f7e6 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:51.903334 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a68934-f2d0-4dc0-a701-7bed3553f7e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:51.907437 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a68934-f2d0-4dc0-a701-7bed3553f7e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:51.926025 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a68934-f2d0-4dc0-a701-7bed3553f7e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:51.927753 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a68934-f2d0-4dc0-a701-7bed3553f7e6 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:51.940334 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:51.945062 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25397441-65d2-455d-aabc-93a2d4d4da85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:51.946107 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25397441-65d2-455d-aabc-93a2d4d4da85 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:51.946561 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25397441-65d2-455d-aabc-93a2d4d4da85 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:51.946985 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25397441-65d2-455d-aabc-93a2d4d4da85 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:51.947819 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25397441-65d2-455d-aabc-93a2d4d4da85 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:51.949354 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25397441-65d2-455d-aabc-93a2d4d4da85 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:51.955494 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-c9a68934-f2d0-4dc0-a701-7bed3553f7e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:26:51.955494 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:51.955494 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:51.955494 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:51.955494 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:51.955494 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:51.955494 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:51.955494 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:51.955494 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:51.955494 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:51.955494 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:51.955494 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:51.955494 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 22:26:51.955494 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 22:26:51.955494 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 22:26:51.956365 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 22:26:51.956365 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:51.956365 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:51.956365 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:51.956365 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:51.956365 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:51.956365 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:51.956365 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:51.956365 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:51.956365 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:51.956365 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:51.956365 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:51.956365 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:51.956365 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:51.956365 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:51.956365 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:51.956365 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:51.957274 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 22:26:51.957274 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 22:26:51.957274 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:26:51.957274 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:26:51.957274 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 411/816] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Jul 20 22:26:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:51.958007 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a68934-f2d0-4dc0-a701-7bed3553f7e6 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:51.964913 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-25397441-65d2-455d-aabc-93a2d4d4da85 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 20 22:26:51.964913 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:51.964913 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 22:26:51.964913 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 22:26:51.964913 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 22:26:51.964913 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 22:26:51.964913 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 22:26:51.964913 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 22:26:51.964913 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 20 22:26:51.964913 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:26:51.964913 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 22:26:51.964913 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:26:51.964913 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:51.964913 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:51.964913 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:51.964913 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:51.965850 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:51.965850 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:51.965850 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:51.965850 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:51.965850 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:51.965850 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:51.965850 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:51.965850 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 22:26:51.965850 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 22:26:51.965850 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 22:26:51.965850 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 22:26:51.965850 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:51.965850 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:51.965850 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:51.965850 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:51.965850 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:51.966678 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:51.966678 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:51.966678 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:51.966678 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:51.966678 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:51.966678 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:51.966678 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:51.966678 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:51.966678 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:51.966678 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:51.966678 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:51.966678 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 22:26:51.966678 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 22:26:51.966678 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 20 22:26:51.966678 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:26:51.966678 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 406/817] 10.222.0.81 () {56 vars in 1129 bytes} [Wed Jul 20 22:26:51 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:51.968328 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25397441-65d2-455d-aabc-93a2d4d4da85 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:52.052575 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:52.071496 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a68934-f2d0-4dc0-a701-7bed3553f7e6 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:52.089317 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25397441-65d2-455d-aabc-93a2d4d4da85 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:52.116589 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395c6378-b6bb-45cb-ad03-604eec95871f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:52.117530 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395c6378-b6bb-45cb-ad03-604eec95871f None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:52.117821 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395c6378-b6bb-45cb-ad03-604eec95871f None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:52.118101 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395c6378-b6bb-45cb-ad03-604eec95871f None admin] PATH_INFO: `/v3/services` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:52.118967 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395c6378-b6bb-45cb-ad03-604eec95871f None admin] RBAC: Adding query filter params (name=network) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:52.119625 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395c6378-b6bb-45cb-ad03-604eec95871f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:52.121286 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395c6378-b6bb-45cb-ad03-604eec95871f None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:52.124078 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062f83e4-7643-42cc-8bfc-c70f2f7f61b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:52.124766 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062f83e4-7643-42cc-8bfc-c70f2f7f61b4 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:52.125058 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062f83e4-7643-42cc-8bfc-c70f2f7f61b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:52.125341 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062f83e4-7643-42cc-8bfc-c70f2f7f61b4 None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:52.126496 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 412/818] 10.222.0.81 () {56 vars in 1112 bytes} [Wed Jul 20 22:26:51 2022] GET /identity/v3/services?name=network => generated 124 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:52.126815 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062f83e4-7643-42cc-8bfc-c70f2f7f61b4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:52.127408 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062f83e4-7643-42cc-8bfc-c70f2f7f61b4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:52.127657 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395c6378-b6bb-45cb-ad03-604eec95871f None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:52.129484 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062f83e4-7643-42cc-8bfc-c70f2f7f61b4 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:52.146874 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 407/819] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Jul 20 22:26:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:52.150661 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062f83e4-7643-42cc-8bfc-c70f2f7f61b4 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:52.233241 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:52.285919 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062f83e4-7643-42cc-8bfc-c70f2f7f61b4 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:52.316299 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:52.326159 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceb1529-e59e-4adb-b207-c4e5e311bdc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:52.327067 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceb1529-e59e-4adb-b207-c4e5e311bdc2 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:52.327502 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceb1529-e59e-4adb-b207-c4e5e311bdc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:52.328073 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceb1529-e59e-4adb-b207-c4e5e311bdc2 None admin] PATH_INFO: `/v3/services` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:52.338196 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ceb1529-e59e-4adb-b207-c4e5e311bdc2 None admin] RBAC: Adding query filter params (type=network) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:52.338196 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ceb1529-e59e-4adb-b207-c4e5e311bdc2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:52.338196 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ceb1529-e59e-4adb-b207-c4e5e311bdc2 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:52.338196 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 408/820] 10.222.0.81 () {56 vars in 1112 bytes} [Wed Jul 20 22:26:52 2022] GET /identity/v3/services?type=network => generated 350 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:52.338196 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceb1529-e59e-4adb-b207-c4e5e311bdc2 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:52.348979 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395c6378-b6bb-45cb-ad03-604eec95871f None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:52.451338 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceb1529-e59e-4adb-b207-c4e5e311bdc2 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:52.451525 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4bd3dc-3b80-4316-96bb-0758ddb468a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:52.452219 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4bd3dc-3b80-4316-96bb-0758ddb468a0 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:52.452603 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4bd3dc-3b80-4316-96bb-0758ddb468a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:52.453002 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4bd3dc-3b80-4316-96bb-0758ddb468a0 None admin] PATH_INFO: `/v3/services` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:52.453980 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4bd3dc-3b80-4316-96bb-0758ddb468a0 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:52.454389 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4bd3dc-3b80-4316-96bb-0758ddb468a0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:52.466982 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4bd3dc-3b80-4316-96bb-0758ddb468a0 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:52.485021 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 413/821] 10.222.0.81 () {56 vars in 1124 bytes} [Wed Jul 20 22:26:52 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 358 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:52.487115 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4bd3dc-3b80-4316-96bb-0758ddb468a0 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:52.503290 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50be01be-0645-4fab-b57c-c5edd2ddc313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:52.504120 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50be01be-0645-4fab-b57c-c5edd2ddc313 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:52.504404 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50be01be-0645-4fab-b57c-c5edd2ddc313 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:52.504681 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50be01be-0645-4fab-b57c-c5edd2ddc313 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:52.515282 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50be01be-0645-4fab-b57c-c5edd2ddc313 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:52.516730 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50be01be-0645-4fab-b57c-c5edd2ddc313 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:52.532740 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-50be01be-0645-4fab-b57c-c5edd2ddc313 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 20 22:26:52.532740 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:52.532740 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:52.532740 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:52.532740 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:52.532740 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:52.532740 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:52.532740 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:52.532740 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:52.532740 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:52.532740 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:52.532740 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:52.532740 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 22:26:52.532740 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 22:26:52.532740 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 22:26:52.532740 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 22:26:52.533748 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:52.533748 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:52.533748 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 22:26:52.533748 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 22:26:52.533748 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 22:26:52.533748 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 22:26:52.533748 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:52.533748 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:52.533748 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:52.533748 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:52.533748 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:52.533748 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:52.533748 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:52.533748 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:52.533748 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:52.533748 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:52.533748 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:52.533748 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:52.534761 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:52.534761 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:52.534761 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 22:26:52.534761 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 22:26:52.534761 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 22:26:52.534761 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 22:26:52.534761 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 22:26:52.534761 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 22:26:52.534761 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 20 22:26:52.534761 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:26:52.539285 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 409/822] 10.222.0.81 () {56 vars in 1093 bytes} [Wed Jul 20 22:26:52 2022] GET /identity/v3/users/nova => generated 82 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:52.540134 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50be01be-0645-4fab-b57c-c5edd2ddc313 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:52.611438 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:52.653141 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50be01be-0645-4fab-b57c-c5edd2ddc313 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:52.697735 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5be779-8bb2-4623-b7df-a075eacbfcee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:52.698790 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5be779-8bb2-4623-b7df-a075eacbfcee None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:52.699248 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5be779-8bb2-4623-b7df-a075eacbfcee None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:52.699447 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5be779-8bb2-4623-b7df-a075eacbfcee None admin] PATH_INFO: `/v3/services` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:52.700282 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5be779-8bb2-4623-b7df-a075eacbfcee None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:52.700713 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5be779-8bb2-4623-b7df-a075eacbfcee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:52.702302 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5be779-8bb2-4623-b7df-a075eacbfcee None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:52.711148 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 410/823] 10.222.0.81 () {56 vars in 1124 bytes} [Wed Jul 20 22:26:52 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:52.712233 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5be779-8bb2-4623-b7df-a075eacbfcee None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:52.722968 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4bd3dc-3b80-4316-96bb-0758ddb468a0 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:52.750083 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:52.763714 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac137556-a90e-479d-ad10-7d0770b7eaf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:52.764910 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac137556-a90e-479d-ad10-7d0770b7eaf6 None admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:52.765197 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac137556-a90e-479d-ad10-7d0770b7eaf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:52.765482 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac137556-a90e-479d-ad10-7d0770b7eaf6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:52.766228 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac137556-a90e-479d-ad10-7d0770b7eaf6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:52.768587 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac137556-a90e-479d-ad10-7d0770b7eaf6 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:52.829991 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5be779-8bb2-4623-b7df-a075eacbfcee None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:52.844120 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:52.867576 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3496bef5-8d03-4ddc-be15-bd6fbd4236dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:52.868314 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3496bef5-8d03-4ddc-be15-bd6fbd4236dd None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:52.868600 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3496bef5-8d03-4ddc-be15-bd6fbd4236dd None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:52.868878 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3496bef5-8d03-4ddc-be15-bd6fbd4236dd None admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:52.869722 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3496bef5-8d03-4ddc-be15-bd6fbd4236dd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:52.870037 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3496bef5-8d03-4ddc-be15-bd6fbd4236dd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:52.871498 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3496bef5-8d03-4ddc-be15-bd6fbd4236dd None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:52.876411 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 414/824] 10.222.0.81 () {60 vars in 1144 bytes} [Wed Jul 20 22:26:52 2022] POST /identity/v3/endpoints => generated 327 bytes in 390 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:26:52.877688 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac137556-a90e-479d-ad10-7d0770b7eaf6 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:52.896508 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 411/825] 10.222.0.81 () {56 vars in 1133 bytes} [Wed Jul 20 22:26:52 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:52.909186 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3496bef5-8d03-4ddc-be15-bd6fbd4236dd None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:53.046645 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac137556-a90e-479d-ad10-7d0770b7eaf6 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:53.090424 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0dc651-f52e-4585-97ce-435e63875534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:53.091632 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0dc651-f52e-4585-97ce-435e63875534 None admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:53.092140 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0dc651-f52e-4585-97ce-435e63875534 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:53.092739 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0dc651-f52e-4585-97ce-435e63875534 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:53.093682 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0dc651-f52e-4585-97ce-435e63875534 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:53.095459 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0dc651-f52e-4585-97ce-435e63875534 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:53.116585 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3496bef5-8d03-4ddc-be15-bd6fbd4236dd None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:53.156229 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fc53d4-9061-470e-a5d8-f930a39d53c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:53.156892 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fc53d4-9061-470e-a5d8-f930a39d53c1 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:53.157180 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fc53d4-9061-470e-a5d8-f930a39d53c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:53.157461 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fc53d4-9061-470e-a5d8-f930a39d53c1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:53.181105 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 415/826] 10.222.0.81 () {60 vars in 1144 bytes} [Wed Jul 20 22:26:52 2022] POST /identity/v3/endpoints => generated 346 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:26:53.219801 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:53.249002 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fc53d4-9061-470e-a5d8-f930a39d53c1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:53.250559 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fc53d4-9061-470e-a5d8-f930a39d53c1 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:53.268840 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2728556-5a58-42b6-a166-5d0d210e3602 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:53.269148 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2728556-5a58-42b6-a166-5d0d210e3602 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:53.269414 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2728556-5a58-42b6-a166-5d0d210e3602 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:53.271191 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 416/827] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:26:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:26:53.276677 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c246cdd-f36c-4cac-a46e-4294fe5d953d None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:53.277026 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c246cdd-f36c-4cac-a46e-4294fe5d953d None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:53.277237 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c246cdd-f36c-4cac-a46e-4294fe5d953d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:53.299819 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-6c246cdd-f36c-4cac-a46e-4294fe5d953d None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:26:53.301089 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-6c246cdd-f36c-4cac-a46e-4294fe5d953d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:26:53.331196 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:53.383655 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 417/828] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:26:53 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 22:26:53.395140 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 412/829] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Jul 20 22:26:52 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 487 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 22:26:53.401192 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c246cdd-f36c-4cac-a46e-4294fe5d953d None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:53.410589 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fc53d4-9061-470e-a5d8-f930a39d53c1 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:53.523531 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c246cdd-f36c-4cac-a46e-4294fe5d953d None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:53.564897 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf47088-3bad-46cc-85fd-840d4a40b72d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:53.565769 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf47088-3bad-46cc-85fd-840d4a40b72d None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:53.566206 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf47088-3bad-46cc-85fd-840d4a40b72d None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:53.566502 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf47088-3bad-46cc-85fd-840d4a40b72d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:53.577778 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf47088-3bad-46cc-85fd-840d4a40b72d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:53.579271 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf47088-3bad-46cc-85fd-840d4a40b72d None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:53.603643 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-8bf47088-3bad-46cc-85fd-840d4a40b72d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:26:53.603643 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:53.603643 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:53.603643 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:53.603643 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:53.603643 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:53.603643 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:53.603643 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:53.603643 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:53.603643 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:53.603643 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:53.603643 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:53.603643 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 22:26:53.603643 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 22:26:53.603643 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 22:26:53.604589 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 22:26:53.604589 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:53.604589 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:53.604589 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:53.604589 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:53.604589 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:53.604589 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:53.604589 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:53.604589 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:53.604589 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:53.604589 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:53.604589 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:53.604589 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:53.604589 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:53.604589 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:53.604589 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:53.604589 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:53.605488 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 22:26:53.605488 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 22:26:53.605488 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:26:53.605488 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:26:53.606227 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:53.610347 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 418/830] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Jul 20 22:26:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:53.612298 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf47088-3bad-46cc-85fd-840d4a40b72d None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:53.633751 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:53.643335 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fc53d4-9061-470e-a5d8-f930a39d53c1 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:53.681486 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82add32d-11d4-4c73-8a79-4a511328ce02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:53.682711 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82add32d-11d4-4c73-8a79-4a511328ce02 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:53.683237 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82add32d-11d4-4c73-8a79-4a511328ce02 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:53.683732 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82add32d-11d4-4c73-8a79-4a511328ce02 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:53.695761 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82add32d-11d4-4c73-8a79-4a511328ce02 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 22:26:53.704503 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-82add32d-11d4-4c73-8a79-4a511328ce02 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 22:26:53.704503 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:53.704503 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:53.704503 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:53.704503 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:53.704503 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:53.704503 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:53.704503 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:53.704503 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:53.704503 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:53.704503 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:53.704503 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:53.704503 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 22:26:53.704503 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 22:26:53.704503 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 22:26:53.704503 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 22:26:53.705607 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 22:26:53.705607 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 22:26:53.705607 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 22:26:53.705607 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 22:26:53.705607 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:53.705607 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:53.705607 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:53.705607 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:53.705607 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:53.705607 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:53.705607 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:53.705607 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:53.705607 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:53.705607 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:53.705607 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:53.705607 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:53.706444 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:53.706444 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:53.706444 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:53.706444 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:53.706444 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 22:26:53.706444 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 22:26:53.706444 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 22:26:53.706444 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 22:26:53.706444 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 22:26:53.706444 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 22:26:53.706444 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 22:26:53.706444 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:26:53.709201 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 413/831] 10.222.0.81 () {56 vars in 1102 bytes} [Wed Jul 20 22:26:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:53.719033 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82add32d-11d4-4c73-8a79-4a511328ce02 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:53.731847 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf47088-3bad-46cc-85fd-840d4a40b72d None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:53.769279 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be297726-49b9-4e01-8eb2-646f1166272d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:53.769965 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be297726-49b9-4e01-8eb2-646f1166272d None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:53.770387 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be297726-49b9-4e01-8eb2-646f1166272d None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:53.770809 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be297726-49b9-4e01-8eb2-646f1166272d None admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:53.771596 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be297726-49b9-4e01-8eb2-646f1166272d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:53.771915 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be297726-49b9-4e01-8eb2-646f1166272d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:53.773757 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be297726-49b9-4e01-8eb2-646f1166272d None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:53.794484 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 419/832] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Jul 20 22:26:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 22:26:53.796367 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be297726-49b9-4e01-8eb2-646f1166272d None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:53.833103 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82add32d-11d4-4c73-8a79-4a511328ce02 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:53.868559 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:53.872777 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6047c4f-4b62-42f1-a5ea-cb5c5e49cfec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:53.873772 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6047c4f-4b62-42f1-a5ea-cb5c5e49cfec None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:53.874809 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6047c4f-4b62-42f1-a5ea-cb5c5e49cfec None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:53.874809 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6047c4f-4b62-42f1-a5ea-cb5c5e49cfec None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:53.875765 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6047c4f-4b62-42f1-a5ea-cb5c5e49cfec None admin] RBAC: Adding query filter params (name=service) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:53.876254 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6047c4f-4b62-42f1-a5ea-cb5c5e49cfec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:53.878513 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6047c4f-4b62-42f1-a5ea-cb5c5e49cfec None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:53.885769 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:53.888106 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 414/833] 10.222.0.81 () {56 vars in 1103 bytes} [Wed Jul 20 22:26:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:53.903036 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6047c4f-4b62-42f1-a5ea-cb5c5e49cfec None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:53.925890 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be297726-49b9-4e01-8eb2-646f1166272d None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:53.975703 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18593d33-898f-4f42-ab17-b2d0fd26d188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:53.977378 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18593d33-898f-4f42-ab17-b2d0fd26d188 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:53.977669 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18593d33-898f-4f42-ab17-b2d0fd26d188 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:53.977955 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18593d33-898f-4f42-ab17-b2d0fd26d188 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:53.987630 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18593d33-898f-4f42-ab17-b2d0fd26d188 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:53.989826 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18593d33-898f-4f42-ab17-b2d0fd26d188 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:54.009294 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6047c4f-4b62-42f1-a5ea-cb5c5e49cfec None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:54.010230 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-18593d33-898f-4f42-ab17-b2d0fd26d188 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 22:26:54.010230 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:54.010230 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:54.010230 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:54.010230 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:54.010230 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:54.010230 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:54.010230 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:54.010230 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:54.010230 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:54.010230 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:54.010230 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:54.010230 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 22:26:54.010230 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 22:26:54.010230 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 22:26:54.011287 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 22:26:54.011287 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:54.011287 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:54.011287 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:54.011287 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:54.011287 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:54.011287 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:54.011287 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:54.011287 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:54.011287 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:54.011287 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:54.011287 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:54.011287 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:54.011287 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:54.011287 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:54.011287 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:54.011287 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:54.012371 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 22:26:54.012371 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 22:26:54.012371 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 22:26:54.012371 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 22:26:54.012371 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 22:26:54.012371 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 22:26:54.012371 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 22:26:54.012371 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:26:54.015096 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 420/834] 10.222.0.81 () {56 vars in 1111 bytes} [Wed Jul 20 22:26:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:54.017908 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18593d33-898f-4f42-ab17-b2d0fd26d188 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:54.058496 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514a66b3-977a-4871-988c-e85df7eabe69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:54.059471 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514a66b3-977a-4871-988c-e85df7eabe69 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:54.059757 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514a66b3-977a-4871-988c-e85df7eabe69 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:54.060032 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514a66b3-977a-4871-988c-e85df7eabe69 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:54.072687 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514a66b3-977a-4871-988c-e85df7eabe69 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:54.074161 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514a66b3-977a-4871-988c-e85df7eabe69 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:54.093800 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-514a66b3-977a-4871-988c-e85df7eabe69 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:26:54.093800 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:54.093800 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:54.093800 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:54.093800 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:54.093800 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:54.093800 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:54.093800 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:54.093800 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:54.093800 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:54.093800 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:54.093800 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:54.093800 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 22:26:54.093800 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 22:26:54.093800 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 22:26:54.094704 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 22:26:54.094704 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:54.094704 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:54.094704 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:54.094704 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:54.094704 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:54.094704 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:54.094704 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:54.094704 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:54.094704 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:54.094704 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:54.094704 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:54.094704 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:54.094704 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:54.094704 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:54.094704 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:54.094704 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:54.096342 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 22:26:54.096342 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 22:26:54.096342 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:26:54.096342 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:26:54.096342 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 415/835] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Jul 20 22:26:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 22:26:54.096342 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9a3579-0aa6-4e72-a651-240493be9dc8 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:54.096720 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9a3579-0aa6-4e72-a651-240493be9dc8 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:54.096720 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9a3579-0aa6-4e72-a651-240493be9dc8 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:54.098144 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 416/836] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:26:54 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:26:54.102650 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9a3579-0aa6-4e72-a651-240493be9dc8 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:54.136851 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18593d33-898f-4f42-ab17-b2d0fd26d188 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:54.154739 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:54.169290 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:54.175345 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275cece6-15c2-4ed7-94de-88888b500fc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:54.176019 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275cece6-15c2-4ed7-94de-88888b500fc1 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:54.176375 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275cece6-15c2-4ed7-94de-88888b500fc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:54.176661 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275cece6-15c2-4ed7-94de-88888b500fc1 None admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:54.177443 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275cece6-15c2-4ed7-94de-88888b500fc1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:54.177754 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275cece6-15c2-4ed7-94de-88888b500fc1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:54.181507 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275cece6-15c2-4ed7-94de-88888b500fc1 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:54.193994 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 421/837] 10.222.0.81 () {56 vars in 1148 bytes} [Wed Jul 20 22:26:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:54.196078 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4813cc55-7f73-4cef-a08e-d258f22ca05c None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:54.196368 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4813cc55-7f73-4cef-a08e-d258f22ca05c None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:54.196649 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4813cc55-7f73-4cef-a08e-d258f22ca05c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:54.216052 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-4813cc55-7f73-4cef-a08e-d258f22ca05c None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:26:54.217782 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-4813cc55-7f73-4cef-a08e-d258f22ca05c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:26:54.220864 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9a3579-0aa6-4e72-a651-240493be9dc8 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:54.254706 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684877ac-f005-4021-88e2-dde9dcf3405e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:54.255405 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684877ac-f005-4021-88e2-dde9dcf3405e None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:54.255696 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684877ac-f005-4021-88e2-dde9dcf3405e None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:54.255980 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684877ac-f005-4021-88e2-dde9dcf3405e None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:54.256701 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684877ac-f005-4021-88e2-dde9dcf3405e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:54.257016 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684877ac-f005-4021-88e2-dde9dcf3405e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:54.258376 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684877ac-f005-4021-88e2-dde9dcf3405e None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:54.275107 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 417/838] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Jul 20 22:26:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:54.275107 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684877ac-f005-4021-88e2-dde9dcf3405e None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:54.303032 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 422/839] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:26:54 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 22:26:54.303980 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4813cc55-7f73-4cef-a08e-d258f22ca05c None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:54.383364 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684877ac-f005-4021-88e2-dde9dcf3405e None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:54.392076 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:54.418800 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4813cc55-7f73-4cef-a08e-d258f22ca05c None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:54.420313 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830c2bc5-ba28-404e-a248-9bfef6aac194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:54.421072 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830c2bc5-ba28-404e-a248-9bfef6aac194 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:54.421430 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830c2bc5-ba28-404e-a248-9bfef6aac194 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:54.421778 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830c2bc5-ba28-404e-a248-9bfef6aac194 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:54.422724 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830c2bc5-ba28-404e-a248-9bfef6aac194 None admin] RBAC: Adding query filter params (role.id=d89b6c92d35a401a8517f1d47f6be843, scope.project.id=385edf2e59e546d090ace36600e29032, user.id=7829324db2574d38b18c749d0559177f) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:54.423168 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830c2bc5-ba28-404e-a248-9bfef6aac194 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:54.425293 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830c2bc5-ba28-404e-a248-9bfef6aac194 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:54.433066 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 418/840] 10.222.0.81 () {56 vars in 1374 bytes} [Wed Jul 20 22:26:54 2022] GET /identity/v3/role_assignments?user.id=7829324db2574d38b18c749d0559177f&scope.project.id=385edf2e59e546d090ace36600e29032&role.id=d89b6c92d35a401a8517f1d47f6be843 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:54.434612 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830c2bc5-ba28-404e-a248-9bfef6aac194 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:54.450807 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:54.464520 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6182f8ce-1b20-4641-a7e4-7e6e4efc4dfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:54.465614 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6182f8ce-1b20-4641-a7e4-7e6e4efc4dfe None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:54.466003 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6182f8ce-1b20-4641-a7e4-7e6e4efc4dfe None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:54.466393 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6182f8ce-1b20-4641-a7e4-7e6e4efc4dfe None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:54.478351 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6182f8ce-1b20-4641-a7e4-7e6e4efc4dfe None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:54.479928 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6182f8ce-1b20-4641-a7e4-7e6e4efc4dfe None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:54.498110 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-6182f8ce-1b20-4641-a7e4-7e6e4efc4dfe None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 22:26:54.498110 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:54.498110 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:54.498110 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:54.498110 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:54.498110 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:54.498110 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:54.498110 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:54.498110 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:54.498110 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:54.498110 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:54.498110 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:54.498110 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 22:26:54.498110 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 22:26:54.498110 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 22:26:54.498110 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 22:26:54.499216 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:54.499216 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:54.499216 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 22:26:54.499216 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 22:26:54.499216 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 22:26:54.499216 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 22:26:54.499216 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:54.499216 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:54.499216 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:54.499216 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:54.499216 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:54.499216 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:54.499216 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:54.499216 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:54.499216 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:54.499216 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:54.499216 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:54.499216 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:54.500239 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:54.500239 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:54.500239 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 22:26:54.500239 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 22:26:54.500239 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 22:26:54.500239 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 22:26:54.500239 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 22:26:54.500239 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 22:26:54.500239 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 22:26:54.500239 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:26:54.504475 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 423/841] 10.222.0.81 () {56 vars in 1117 bytes} [Wed Jul 20 22:26:54 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:54.505724 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6182f8ce-1b20-4641-a7e4-7e6e4efc4dfe None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:54.608796 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830c2bc5-ba28-404e-a248-9bfef6aac194 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:54.644852 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6182f8ce-1b20-4641-a7e4-7e6e4efc4dfe None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:54.656905 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c9f53e-6e90-466c-9127-4c0ccc1056ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:54.658058 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c9f53e-6e90-466c-9127-4c0ccc1056ef None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:54.659534 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c9f53e-6e90-466c-9127-4c0ccc1056ef None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:54.659534 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c9f53e-6e90-466c-9127-4c0ccc1056ef None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:54.675242 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c9f53e-6e90-466c-9127-4c0ccc1056ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 22:26:54.683868 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-b3c9f53e-6e90-466c-9127-4c0ccc1056ef None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 22:26:54.683868 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:54.683868 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:54.683868 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:54.683868 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:54.683868 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:54.683868 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:54.683868 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:54.683868 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:54.683868 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:54.683868 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:54.683868 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:54.683868 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 22:26:54.683868 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 22:26:54.683868 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 22:26:54.683868 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 22:26:54.684798 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 22:26:54.684798 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 22:26:54.684798 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 22:26:54.684798 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 22:26:54.684798 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:54.684798 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:54.684798 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:54.684798 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:54.684798 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:54.684798 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:54.684798 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:54.684798 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:54.684798 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:54.684798 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:54.684798 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:54.684798 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:54.685617 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:54.685617 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:54.685617 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:54.685617 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:54.685617 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 22:26:54.685617 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 22:26:54.685617 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 22:26:54.685617 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 22:26:54.685617 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 22:26:54.685617 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 22:26:54.685617 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 22:26:54.685617 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:26:54.685617 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 419/842] 10.222.0.81 () {56 vars in 1096 bytes} [Wed Jul 20 22:26:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:54.687485 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd909990-4ebe-4d1c-bde2-c77ae00b6c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:54.688164 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd909990-4ebe-4d1c-bde2-c77ae00b6c8f None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:54.688449 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd909990-4ebe-4d1c-bde2-c77ae00b6c8f None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:54.688726 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd909990-4ebe-4d1c-bde2-c77ae00b6c8f None admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:54.690182 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd909990-4ebe-4d1c-bde2-c77ae00b6c8f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:54.690498 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd909990-4ebe-4d1c-bde2-c77ae00b6c8f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:54.691746 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c9f53e-6e90-466c-9127-4c0ccc1056ef None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:54.692910 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd909990-4ebe-4d1c-bde2-c77ae00b6c8f None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:54.713341 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 424/843] 10.222.0.81 () {56 vars in 1149 bytes} [Wed Jul 20 22:26:54 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:54.720147 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd909990-4ebe-4d1c-bde2-c77ae00b6c8f None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:54.731210 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:54.766502 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:54.807942 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c9f53e-6e90-466c-9127-4c0ccc1056ef None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:54.830913 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd909990-4ebe-4d1c-bde2-c77ae00b6c8f None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:54.844199 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b98f27d-6215-42f1-bfe9-45eca5d9dc2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:54.844929 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b98f27d-6215-42f1-bfe9-45eca5d9dc2e None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:54.845162 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b98f27d-6215-42f1-bfe9-45eca5d9dc2e None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:54.845442 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b98f27d-6215-42f1-bfe9-45eca5d9dc2e None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:54.846265 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b98f27d-6215-42f1-bfe9-45eca5d9dc2e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:54.846580 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b98f27d-6215-42f1-bfe9-45eca5d9dc2e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:54.847992 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b98f27d-6215-42f1-bfe9-45eca5d9dc2e None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:54.857181 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 420/844] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Jul 20 22:26:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 22:26:54.864511 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b98f27d-6215-42f1-bfe9-45eca5d9dc2e None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:54.871381 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798c17ef-dbfa-4c7f-b4ee-68b91d6f9219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:54.872237 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798c17ef-dbfa-4c7f-b4ee-68b91d6f9219 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:54.872703 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798c17ef-dbfa-4c7f-b4ee-68b91d6f9219 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:54.873144 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798c17ef-dbfa-4c7f-b4ee-68b91d6f9219 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:54.947943 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798c17ef-dbfa-4c7f-b4ee-68b91d6f9219 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:54.949720 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798c17ef-dbfa-4c7f-b4ee-68b91d6f9219 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:54.966684 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:54.976753 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b98f27d-6215-42f1-bfe9-45eca5d9dc2e None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:54.995099 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:55.013374 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31da6c2c-ba5c-41c1-938b-b6ff129bb1dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:55.014194 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31da6c2c-ba5c-41c1-938b-b6ff129bb1dc None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:55.014570 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31da6c2c-ba5c-41c1-938b-b6ff129bb1dc None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:55.014748 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31da6c2c-ba5c-41c1-938b-b6ff129bb1dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:55.026736 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31da6c2c-ba5c-41c1-938b-b6ff129bb1dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:55.028327 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31da6c2c-ba5c-41c1-938b-b6ff129bb1dc None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:55.047923 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-31da6c2c-ba5c-41c1-938b-b6ff129bb1dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:26:55.047923 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:55.047923 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:55.047923 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:55.047923 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:55.047923 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:55.047923 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:55.047923 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:55.047923 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:55.047923 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:55.047923 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:55.047923 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:55.047923 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 22:26:55.047923 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 22:26:55.047923 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 22:26:55.048928 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 22:26:55.048928 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:55.048928 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:55.048928 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:55.048928 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:55.048928 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:55.048928 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:55.048928 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:55.048928 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:55.048928 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:55.048928 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:55.048928 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:55.048928 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:55.048928 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:55.048928 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:55.048928 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:55.048928 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:55.049817 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 22:26:55.049817 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 22:26:55.049817 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:26:55.049817 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:26:55.049817 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 421/845] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Jul 20 22:26:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:55.055524 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31da6c2c-ba5c-41c1-938b-b6ff129bb1dc None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:55.074734 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 425/846] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Jul 20 22:26:54 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 20 22:26:55.076050 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798c17ef-dbfa-4c7f-b4ee-68b91d6f9219 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:55.198933 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31da6c2c-ba5c-41c1-938b-b6ff129bb1dc None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:55.216903 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798c17ef-dbfa-4c7f-b4ee-68b91d6f9219 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:55.244091 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2745e4-9152-4045-a103-b3804f83c6c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:55.244765 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2745e4-9152-4045-a103-b3804f83c6c2 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:55.245049 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2745e4-9152-4045-a103-b3804f83c6c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:55.245326 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2745e4-9152-4045-a103-b3804f83c6c2 None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:55.246047 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2745e4-9152-4045-a103-b3804f83c6c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:55.246352 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2745e4-9152-4045-a103-b3804f83c6c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:55.247745 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2745e4-9152-4045-a103-b3804f83c6c2 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:55.261112 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 422/847] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Jul 20 22:26:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:55.262786 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74420166-9739-44ba-9579-3218265ebf24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:55.263636 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74420166-9739-44ba-9579-3218265ebf24 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:55.263924 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74420166-9739-44ba-9579-3218265ebf24 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:55.264206 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74420166-9739-44ba-9579-3218265ebf24 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:55.266237 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2745e4-9152-4045-a103-b3804f83c6c2 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:55.270174 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:55.273353 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:55.276113 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74420166-9739-44ba-9579-3218265ebf24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:55.277747 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74420166-9739-44ba-9579-3218265ebf24 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:55.297110 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-74420166-9739-44ba-9579-3218265ebf24 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:26:55.297110 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:55.297110 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:55.297110 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:55.297110 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:55.297110 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:55.297110 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:55.297110 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:55.297110 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:55.297110 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:55.297110 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:55.297110 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:55.297110 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 22:26:55.297110 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 22:26:55.297110 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 22:26:55.298221 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 22:26:55.298221 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:55.298221 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:55.298221 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:55.298221 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:55.298221 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:55.298221 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:55.298221 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:55.298221 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:55.298221 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:55.298221 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:55.298221 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:55.298221 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:55.298221 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:55.298221 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:55.298221 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:55.298221 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:55.299722 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 22:26:55.299722 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 22:26:55.299722 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:26:55.299722 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:26:55.303127 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 426/848] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Jul 20 22:26:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 22:26:55.304863 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74420166-9739-44ba-9579-3218265ebf24 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:55.376092 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2745e4-9152-4045-a103-b3804f83c6c2 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:55.410501 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74420166-9739-44ba-9579-3218265ebf24 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:55.413040 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50859ffc-1a31-401e-ab05-41340684182c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:55.414097 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50859ffc-1a31-401e-ab05-41340684182c None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:55.414588 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50859ffc-1a31-401e-ab05-41340684182c None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:55.415109 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50859ffc-1a31-401e-ab05-41340684182c None admin] PATH_INFO: `/v3/users/swift` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:55.424850 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50859ffc-1a31-401e-ab05-41340684182c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:55.426435 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50859ffc-1a31-401e-ab05-41340684182c None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:55.442548 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-50859ffc-1a31-401e-ab05-41340684182c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 20 22:26:55.442548 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:55.442548 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:55.442548 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:55.442548 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:55.442548 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:55.442548 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:55.442548 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:55.442548 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:55.442548 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:55.442548 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:55.442548 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:55.442548 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 22:26:55.442548 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 22:26:55.442548 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 22:26:55.442548 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 22:26:55.443732 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:55.443732 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:55.443732 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 22:26:55.443732 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 22:26:55.443732 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 22:26:55.443732 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 22:26:55.443732 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:55.443732 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:55.443732 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:55.443732 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:55.443732 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:55.443732 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:55.443732 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:55.443732 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:55.443732 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:55.443732 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:55.443732 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:55.443732 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:55.444951 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:55.444951 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:55.444951 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 22:26:55.444951 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 22:26:55.444951 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 22:26:55.444951 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 22:26:55.444951 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 22:26:55.444951 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 22:26:55.444951 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 20 22:26:55.444951 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:26:55.446129 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 423/849] 10.222.0.81 () {56 vars in 1096 bytes} [Wed Jul 20 22:26:55 2022] GET /identity/v3/users/swift => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:55.447266 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c33316-be42-48f9-8356-c52abb24d79c None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:55.447551 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c33316-be42-48f9-8356-c52abb24d79c None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:55.447825 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c33316-be42-48f9-8356-c52abb24d79c None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:55.449259 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 424/850] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:26:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:26:55.453701 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c33316-be42-48f9-8356-c52abb24d79c None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:55.455071 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61675392-e613-422e-b3f4-ae7ff49b53af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:55.455750 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61675392-e613-422e-b3f4-ae7ff49b53af None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:55.456041 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61675392-e613-422e-b3f4-ae7ff49b53af None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:55.456324 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61675392-e613-422e-b3f4-ae7ff49b53af None admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:55.457055 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61675392-e613-422e-b3f4-ae7ff49b53af None admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:55.457425 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61675392-e613-422e-b3f4-ae7ff49b53af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:55.458834 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61675392-e613-422e-b3f4-ae7ff49b53af None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:55.471591 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 427/851] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Jul 20 22:26:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 22:26:55.472701 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b7f672-c20b-458b-b0a7-f54133199f5e None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:55.472959 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b7f672-c20b-458b-b0a7-f54133199f5e None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:55.473240 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b7f672-c20b-458b-b0a7-f54133199f5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:55.490095 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-06b7f672-c20b-458b-b0a7-f54133199f5e None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:26:55.491330 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-06b7f672-c20b-458b-b0a7-f54133199f5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:26:55.527354 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:55.542077 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:55.562023 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 428/852] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:26:55 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 22:26:55.563068 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b7f672-c20b-458b-b0a7-f54133199f5e None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:55.568684 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c33316-be42-48f9-8356-c52abb24d79c None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:55.605276 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f1481b-fb37-48da-859b-40aef2d2ef06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:55.605989 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f1481b-fb37-48da-859b-40aef2d2ef06 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:55.606274 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f1481b-fb37-48da-859b-40aef2d2ef06 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:55.606549 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f1481b-fb37-48da-859b-40aef2d2ef06 None admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:55.607478 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f1481b-fb37-48da-859b-40aef2d2ef06 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:55.607766 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f1481b-fb37-48da-859b-40aef2d2ef06 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:55.609182 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f1481b-fb37-48da-859b-40aef2d2ef06 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:55.627817 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 425/853] 10.222.0.81 () {56 vars in 1135 bytes} [Wed Jul 20 22:26:55 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:55.628436 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f1481b-fb37-48da-859b-40aef2d2ef06 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:55.670798 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b7f672-c20b-458b-b0a7-f54133199f5e None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:55.711232 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f13bbe2-d745-4128-81bc-e1d4861ee586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:55.711987 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f13bbe2-d745-4128-81bc-e1d4861ee586 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:55.712631 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f13bbe2-d745-4128-81bc-e1d4861ee586 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:55.712916 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f13bbe2-d745-4128-81bc-e1d4861ee586 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:55.721761 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f13bbe2-d745-4128-81bc-e1d4861ee586 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:55.723315 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f13bbe2-d745-4128-81bc-e1d4861ee586 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:55.734382 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f1481b-fb37-48da-859b-40aef2d2ef06 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:55.738683 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-7f13bbe2-d745-4128-81bc-e1d4861ee586 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 22:26:55.738683 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:55.738683 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:55.738683 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:55.738683 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:55.738683 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:55.738683 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:55.738683 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:55.738683 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:55.738683 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:55.738683 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:55.738683 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:55.738683 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 22:26:55.738683 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 22:26:55.738683 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 22:26:55.739607 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 22:26:55.739607 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:55.739607 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:55.739607 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:55.739607 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:55.739607 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:55.739607 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:55.739607 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:55.739607 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:55.739607 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:55.739607 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:55.739607 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:55.739607 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:55.739607 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:55.739607 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:55.739607 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:55.739607 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:55.740470 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 22:26:55.740470 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 22:26:55.740470 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 22:26:55.740470 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 22:26:55.740470 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 22:26:55.740470 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 22:26:55.740470 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 22:26:55.740470 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:26:55.744560 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 429/854] 10.222.0.81 () {56 vars in 1111 bytes} [Wed Jul 20 22:26:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:55.745337 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f13bbe2-d745-4128-81bc-e1d4861ee586 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:55.759158 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:55.771555 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34ab641-0466-4706-8937-72af97140455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:55.772379 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34ab641-0466-4706-8937-72af97140455 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:55.772656 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34ab641-0466-4706-8937-72af97140455 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:55.772931 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34ab641-0466-4706-8937-72af97140455 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:55.776315 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34ab641-0466-4706-8937-72af97140455 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:55.776315 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34ab641-0466-4706-8937-72af97140455 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:55.788517 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-a34ab641-0466-4706-8937-72af97140455 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 20 22:26:55.788517 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:55.788517 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 22:26:55.788517 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 22:26:55.788517 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 22:26:55.788517 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 22:26:55.788517 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 22:26:55.788517 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 22:26:55.788517 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 20 22:26:55.788517 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:26:55.788517 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 22:26:55.788517 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:26:55.788517 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:55.788517 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:55.788517 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:55.788517 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:55.788517 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:55.789784 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:55.789784 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:55.789784 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:55.789784 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:55.789784 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:55.789784 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:55.789784 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 22:26:55.789784 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 22:26:55.789784 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 22:26:55.789784 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 22:26:55.789784 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:55.789784 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:55.789784 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:55.789784 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:55.789784 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:55.789784 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:55.790797 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:55.790797 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:55.790797 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:55.790797 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:55.790797 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:55.790797 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:55.790797 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:55.790797 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:55.790797 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:55.790797 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:55.790797 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 22:26:55.790797 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 22:26:55.790797 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 20 22:26:55.790797 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:26:55.795234 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 426/855] 10.222.0.81 () {56 vars in 1114 bytes} [Wed Jul 20 22:26:55 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:55.796299 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34ab641-0466-4706-8937-72af97140455 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:55.797665 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:55.862901 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f13bbe2-d745-4128-81bc-e1d4861ee586 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:55.899976 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7e573c-5ff5-426e-8c24-36bfa4b1b576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:55.900636 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7e573c-5ff5-426e-8c24-36bfa4b1b576 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:55.900952 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7e573c-5ff5-426e-8c24-36bfa4b1b576 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:55.901216 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7e573c-5ff5-426e-8c24-36bfa4b1b576 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:55.907323 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34ab641-0466-4706-8937-72af97140455 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:55.945187 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb7c540-7930-47ab-afca-ff65c155f238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:55.945856 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb7c540-7930-47ab-afca-ff65c155f238 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:55.946147 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb7c540-7930-47ab-afca-ff65c155f238 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:55.946428 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb7c540-7930-47ab-afca-ff65c155f238 None admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:55.947286 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb7c540-7930-47ab-afca-ff65c155f238 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:55.947561 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb7c540-7930-47ab-afca-ff65c155f238 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:55.948980 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb7c540-7930-47ab-afca-ff65c155f238 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:55.961729 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 427/856] 10.222.0.81 () {56 vars in 1148 bytes} [Wed Jul 20 22:26:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:55.962311 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb7c540-7930-47ab-afca-ff65c155f238 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:55.978022 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7e573c-5ff5-426e-8c24-36bfa4b1b576 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:55.979533 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7e573c-5ff5-426e-8c24-36bfa4b1b576 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:56.010622 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:56.023985 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:56.067273 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb7c540-7930-47ab-afca-ff65c155f238 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:56.082101 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 430/857] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Jul 20 22:26:55 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 22:26:56.082897 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7e573c-5ff5-426e-8c24-36bfa4b1b576 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:56.108745 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd4f4d1-5ae5-4284-9763-cbf7ef502487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:56.109423 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd4f4d1-5ae5-4284-9763-cbf7ef502487 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:56.109850 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd4f4d1-5ae5-4284-9763-cbf7ef502487 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:56.110135 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd4f4d1-5ae5-4284-9763-cbf7ef502487 None admin] PATH_INFO: `/v3/services` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:56.111394 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd4f4d1-5ae5-4284-9763-cbf7ef502487 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:56.111844 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd4f4d1-5ae5-4284-9763-cbf7ef502487 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:56.113674 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd4f4d1-5ae5-4284-9763-cbf7ef502487 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:56.123837 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 428/858] 10.222.0.81 () {56 vars in 1114 bytes} [Wed Jul 20 22:26:55 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:56.123837 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd4f4d1-5ae5-4284-9763-cbf7ef502487 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:56.199473 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7e573c-5ff5-426e-8c24-36bfa4b1b576 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:56.232012 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd4f4d1-5ae5-4284-9763-cbf7ef502487 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:56.238243 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776e654b-dcaa-4d9a-b355-93735229265a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:56.239573 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776e654b-dcaa-4d9a-b355-93735229265a None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:56.240051 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776e654b-dcaa-4d9a-b355-93735229265a None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:56.240517 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776e654b-dcaa-4d9a-b355-93735229265a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:56.241929 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776e654b-dcaa-4d9a-b355-93735229265a None admin] RBAC: Adding query filter params (role.id=9b4a83e2eca44a8da9f356bc40f8788a, scope.project.id=385edf2e59e546d090ace36600e29032, user.id=90915e1264324f16bc7bae7e46e0c97b) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:56.242436 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776e654b-dcaa-4d9a-b355-93735229265a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:56.242743 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:56.244660 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776e654b-dcaa-4d9a-b355-93735229265a None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:56.254864 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 431/859] 10.222.0.81 () {56 vars in 1374 bytes} [Wed Jul 20 22:26:56 2022] GET /identity/v3/role_assignments?user.id=90915e1264324f16bc7bae7e46e0c97b&scope.project.id=385edf2e59e546d090ace36600e29032&role.id=9b4a83e2eca44a8da9f356bc40f8788a => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:56.254864 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776e654b-dcaa-4d9a-b355-93735229265a None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:56.266184 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a79c10c-5c94-403a-96df-a62e493381b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:56.267043 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a79c10c-5c94-403a-96df-a62e493381b8 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:56.267342 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a79c10c-5c94-403a-96df-a62e493381b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:56.267622 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a79c10c-5c94-403a-96df-a62e493381b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:56.272856 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:56.279346 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a79c10c-5c94-403a-96df-a62e493381b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:56.280808 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a79c10c-5c94-403a-96df-a62e493381b8 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:56.301464 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-7a79c10c-5c94-403a-96df-a62e493381b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:26:56.301464 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:56.301464 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:56.301464 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:56.301464 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:56.301464 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:56.301464 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:56.301464 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:56.301464 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:56.301464 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:56.301464 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:56.301464 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:56.301464 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 22:26:56.301464 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 22:26:56.301464 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 22:26:56.302698 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 22:26:56.302698 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:56.302698 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:56.302698 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:56.302698 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:56.302698 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:56.302698 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:56.302698 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:56.302698 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:56.302698 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:56.302698 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:56.302698 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:56.302698 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:56.302698 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:56.302698 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:56.302698 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:56.302698 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:56.303654 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 22:26:56.303654 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 22:26:56.303654 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:26:56.303654 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:26:56.303893 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 429/860] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Jul 20 22:26:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:56.309492 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a79c10c-5c94-403a-96df-a62e493381b8 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:56.411133 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776e654b-dcaa-4d9a-b355-93735229265a None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:56.449216 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a79c10c-5c94-403a-96df-a62e493381b8 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:56.449557 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe56519-3c34-4b0f-9d4f-8b8480f477c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:56.450271 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe56519-3c34-4b0f-9d4f-8b8480f477c4 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:56.450566 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe56519-3c34-4b0f-9d4f-8b8480f477c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:56.450853 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe56519-3c34-4b0f-9d4f-8b8480f477c4 None admin] PATH_INFO: `/v3/services` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:56.451609 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe56519-3c34-4b0f-9d4f-8b8480f477c4 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:56.451923 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe56519-3c34-4b0f-9d4f-8b8480f477c4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:56.454123 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe56519-3c34-4b0f-9d4f-8b8480f477c4 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:56.459090 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 432/861] 10.222.0.81 () {56 vars in 1114 bytes} [Wed Jul 20 22:26:56 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:56.504409 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245bb088-6973-464c-92af-efa1fed3b0a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:56.505029 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245bb088-6973-464c-92af-efa1fed3b0a4 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:56.505312 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245bb088-6973-464c-92af-efa1fed3b0a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:56.505587 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245bb088-6973-464c-92af-efa1fed3b0a4 None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:56.506335 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245bb088-6973-464c-92af-efa1fed3b0a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:56.506647 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245bb088-6973-464c-92af-efa1fed3b0a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:56.508135 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245bb088-6973-464c-92af-efa1fed3b0a4 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:56.523152 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 430/862] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Jul 20 22:26:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 22:26:56.529232 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe56519-3c34-4b0f-9d4f-8b8480f477c4 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:56.616486 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:56.648444 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe56519-3c34-4b0f-9d4f-8b8480f477c4 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:56.684483 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ba0a67-03f6-4c35-8328-ee57e22eb7ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:56.685300 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ba0a67-03f6-4c35-8328-ee57e22eb7ee None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:56.685585 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ba0a67-03f6-4c35-8328-ee57e22eb7ee None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:56.685863 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ba0a67-03f6-4c35-8328-ee57e22eb7ee None admin] PATH_INFO: `/v3/projects/service` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:56.693168 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ba0a67-03f6-4c35-8328-ee57e22eb7ee None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:56.694621 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ba0a67-03f6-4c35-8328-ee57e22eb7ee None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:56.709426 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-f8ba0a67-03f6-4c35-8328-ee57e22eb7ee None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 22:26:56.709426 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:56.709426 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:56.709426 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:56.709426 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:56.709426 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:56.709426 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:56.709426 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:56.709426 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:56.709426 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:56.709426 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:56.709426 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:56.709426 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 22:26:56.709426 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 22:26:56.709426 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 22:26:56.710399 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 22:26:56.710399 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:56.710399 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:56.710399 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:56.710399 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:56.710399 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:56.710399 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:56.710399 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:56.710399 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:56.710399 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:56.710399 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:56.710399 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:56.710399 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:56.710399 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:56.710399 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:56.710399 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:56.710399 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:56.711411 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 22:26:56.711411 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 22:26:56.711411 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 22:26:56.711411 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 22:26:56.711411 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 22:26:56.711411 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 22:26:56.711411 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 22:26:56.711411 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:26:56.711411 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 433/863] 10.222.0.81 () {56 vars in 1111 bytes} [Wed Jul 20 22:26:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:56.716567 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245bb088-6973-464c-92af-efa1fed3b0a4 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:56.732580 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff11981a-e386-473d-977e-586e77aacc57 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:56.733319 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff11981a-e386-473d-977e-586e77aacc57 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:56.733799 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff11981a-e386-473d-977e-586e77aacc57 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:56.735860 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 434/864] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:26:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 22:26:56.742184 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19137795-314f-4f42-9052-c936dd20aa9d None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:56.742731 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19137795-314f-4f42-9052-c936dd20aa9d None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:56.743292 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19137795-314f-4f42-9052-c936dd20aa9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:56.744746 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:56.763250 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-19137795-314f-4f42-9052-c936dd20aa9d None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:26:56.764737 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-19137795-314f-4f42-9052-c936dd20aa9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:26:56.826601 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245bb088-6973-464c-92af-efa1fed3b0a4 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:56.847082 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 435/865] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:26:56 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 22:26:56.864132 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9ff756-2b3a-423d-bce6-4e66b3efb96c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:56.864847 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9ff756-2b3a-423d-bce6-4e66b3efb96c None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:56.865131 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9ff756-2b3a-423d-bce6-4e66b3efb96c None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:56.865405 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9ff756-2b3a-423d-bce6-4e66b3efb96c None admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:56.866018 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19137795-314f-4f42-9052-c936dd20aa9d None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:56.866188 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9ff756-2b3a-423d-bce6-4e66b3efb96c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:56.866493 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9ff756-2b3a-423d-bce6-4e66b3efb96c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:56.867953 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9ff756-2b3a-423d-bce6-4e66b3efb96c None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:56.887140 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 431/866] 10.222.0.81 () {56 vars in 1148 bytes} [Wed Jul 20 22:26:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:56.888863 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9ff756-2b3a-423d-bce6-4e66b3efb96c None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:56.935596 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:56.978978 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19137795-314f-4f42-9052-c936dd20aa9d None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:56.999907 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:57.019032 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9ff756-2b3a-423d-bce6-4e66b3efb96c None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:57.027702 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634a29f9-08bd-4bc4-a0ec-bf20c7bd4f64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:57.029326 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634a29f9-08bd-4bc4-a0ec-bf20c7bd4f64 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:57.030976 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634a29f9-08bd-4bc4-a0ec-bf20c7bd4f64 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:57.031613 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634a29f9-08bd-4bc4-a0ec-bf20c7bd4f64 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:57.032556 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634a29f9-08bd-4bc4-a0ec-bf20c7bd4f64 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:57.034867 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634a29f9-08bd-4bc4-a0ec-bf20c7bd4f64 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:57.054090 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-634a29f9-08bd-4bc4-a0ec-bf20c7bd4f64 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 20 22:26:57.054090 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:57.054090 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 22:26:57.054090 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 22:26:57.054090 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 22:26:57.054090 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 22:26:57.054090 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 22:26:57.054090 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 22:26:57.054090 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 20 22:26:57.054090 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:26:57.054090 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 22:26:57.054090 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:26:57.054090 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:57.054090 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:57.054090 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:57.054090 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:57.055342 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:57.055342 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:57.055342 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:57.055342 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:57.055342 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:57.055342 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:57.055342 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:57.055342 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 22:26:57.055342 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 22:26:57.055342 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 22:26:57.055342 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 22:26:57.055342 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:57.055342 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:57.055342 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:57.055342 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:57.055342 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:57.056390 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:57.056390 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:57.056390 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:57.056390 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:57.056390 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:57.056390 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:57.056390 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:57.056390 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:57.056390 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:57.056390 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:57.056390 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:57.056390 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 22:26:57.056390 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 22:26:57.056390 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 20 22:26:57.056390 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:26:57.058684 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6142c5-cbdb-42ea-b659-25d7e767a707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:57.059028 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 436/867] 10.222.0.81 () {56 vars in 1132 bytes} [Wed Jul 20 22:26:56 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 22:26:57.063904 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6142c5-cbdb-42ea-b659-25d7e767a707 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:57.063904 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6142c5-cbdb-42ea-b659-25d7e767a707 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:57.063904 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6142c5-cbdb-42ea-b659-25d7e767a707 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:57.063904 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6142c5-cbdb-42ea-b659-25d7e767a707 None admin] RBAC: Adding query filter params (role.id=d89b6c92d35a401a8517f1d47f6be843, scope.project.id=385edf2e59e546d090ace36600e29032, user.id=70e97436a08543a180225561ef8bc817) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:57.063904 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6142c5-cbdb-42ea-b659-25d7e767a707 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:57.063904 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6142c5-cbdb-42ea-b659-25d7e767a707 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:57.066072 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634a29f9-08bd-4bc4-a0ec-bf20c7bd4f64 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:57.079195 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 432/868] 10.222.0.81 () {56 vars in 1374 bytes} [Wed Jul 20 22:26:56 2022] GET /identity/v3/role_assignments?user.id=70e97436a08543a180225561ef8bc817&scope.project.id=385edf2e59e546d090ace36600e29032&role.id=d89b6c92d35a401a8517f1d47f6be843 => generated 608 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:57.186786 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634a29f9-08bd-4bc4-a0ec-bf20c7bd4f64 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:57.216621 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:57.228343 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07d9ab8-8441-4371-b589-152163a9573e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:57.231787 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07d9ab8-8441-4371-b589-152163a9573e None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:57.231787 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07d9ab8-8441-4371-b589-152163a9573e None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:57.231787 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07d9ab8-8441-4371-b589-152163a9573e None admin] PATH_INFO: `/v3/services` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:57.231787 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07d9ab8-8441-4371-b589-152163a9573e None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:57.231787 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07d9ab8-8441-4371-b589-152163a9573e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:57.232219 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07d9ab8-8441-4371-b589-152163a9573e None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:57.237152 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 437/869] 10.222.0.81 () {56 vars in 1126 bytes} [Wed Jul 20 22:26:57 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:57.243521 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6142c5-cbdb-42ea-b659-25d7e767a707 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:57.350512 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6142c5-cbdb-42ea-b659-25d7e767a707 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:57.384556 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cb380c-4fe8-4ede-8a5c-137f615205b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:57.385240 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cb380c-4fe8-4ede-8a5c-137f615205b1 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:57.385528 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cb380c-4fe8-4ede-8a5c-137f615205b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:57.385808 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cb380c-4fe8-4ede-8a5c-137f615205b1 None admin] PATH_INFO: `/v3/services` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:57.386522 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cb380c-4fe8-4ede-8a5c-137f615205b1 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:57.386892 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cb380c-4fe8-4ede-8a5c-137f615205b1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:57.388292 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cb380c-4fe8-4ede-8a5c-137f615205b1 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:57.393765 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 433/870] 10.222.0.81 () {56 vars in 1126 bytes} [Wed Jul 20 22:26:57 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:58.543060 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddb63df-4195-4e8e-8727-94f671c3eea1 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:58.543910 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddb63df-4195-4e8e-8727-94f671c3eea1 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:58.544341 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddb63df-4195-4e8e-8727-94f671c3eea1 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:58.546095 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 438/871] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:26:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:26:58.552901 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fa9876-97fa-4922-9e13-a5dd1978e175 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:58.553739 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fa9876-97fa-4922-9e13-a5dd1978e175 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:58.554231 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fa9876-97fa-4922-9e13-a5dd1978e175 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:58.568159 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e67c9d6-6b30-4b6b-b999-010ec2e7dd73 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:58.568159 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e67c9d6-6b30-4b6b-b999-010ec2e7dd73 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:58.568159 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e67c9d6-6b30-4b6b-b999-010ec2e7dd73 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:58.569737 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 439/872] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:26:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:26:58.574740 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d5ae11-6af1-495b-aaae-03441fc32c56 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:58.575055 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d5ae11-6af1-495b-aaae-03441fc32c56 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:58.575347 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d5ae11-6af1-495b-aaae-03441fc32c56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:58.577882 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-94fa9876-97fa-4922-9e13-a5dd1978e175 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:26:58.579356 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-94fa9876-97fa-4922-9e13-a5dd1978e175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:26:58.583479 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:58.593739 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-52d5ae11-6af1-495b-aaae-03441fc32c56 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:26:58.599229 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-52d5ae11-6af1-495b-aaae-03441fc32c56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:26:58.662355 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 434/873] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:26:58 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 22:26:58.678965 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 440/874] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:26:58 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 22:26:58.684761 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fa9876-97fa-4922-9e13-a5dd1978e175 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:58.702508 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d5ae11-6af1-495b-aaae-03441fc32c56 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:58.799990 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fa9876-97fa-4922-9e13-a5dd1978e175 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:58.827675 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d5ae11-6af1-495b-aaae-03441fc32c56 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:58.845389 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f007a4c1-6709-4997-b21a-b8be26a42c3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:58.846723 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f007a4c1-6709-4997-b21a-b8be26a42c3b None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:58.847311 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f007a4c1-6709-4997-b21a-b8be26a42c3b None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:58.847596 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f007a4c1-6709-4997-b21a-b8be26a42c3b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:58.849522 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:58.858515 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:58.863886 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f007a4c1-6709-4997-b21a-b8be26a42c3b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 22:26:58.870932 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4970f941-719c-40f0-815a-63ed1d59f882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:58.871706 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4970f941-719c-40f0-815a-63ed1d59f882 None admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:58.871993 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4970f941-719c-40f0-815a-63ed1d59f882 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:58.872273 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4970f941-719c-40f0-815a-63ed1d59f882 None admin] PATH_INFO: `/v3/services` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:58.873156 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4970f941-719c-40f0-815a-63ed1d59f882 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:58.874564 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-f007a4c1-6709-4997-b21a-b8be26a42c3b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 22:26:58.874564 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:58.874564 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:58.874564 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:58.874564 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:58.874564 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:58.874564 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:58.874564 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:58.874564 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:58.874564 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:58.874564 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:58.874564 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:58.874564 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 22:26:58.874564 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 22:26:58.874564 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 22:26:58.874564 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 22:26:58.875618 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 22:26:58.875618 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 22:26:58.875618 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 22:26:58.875618 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 22:26:58.875618 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:58.875618 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:58.875618 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:58.875618 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:58.875618 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:58.875618 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:58.875618 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:58.875618 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:58.875618 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:58.875618 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:58.875618 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:58.875618 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:58.877052 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:58.877052 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:58.877052 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:58.877052 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:58.877052 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 22:26:58.877052 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 22:26:58.877052 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 22:26:58.877052 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 22:26:58.877052 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 22:26:58.877052 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 22:26:58.877052 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 22:26:58.877052 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:26:58.877705 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4970f941-719c-40f0-815a-63ed1d59f882 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:58.879823 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 435/875] 10.222.0.81 () {56 vars in 1120 bytes} [Wed Jul 20 22:26:58 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:58.885917 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f007a4c1-6709-4997-b21a-b8be26a42c3b None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:58.942752 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 441/876] 10.222.0.81 () {60 vars in 1141 bytes} [Wed Jul 20 22:26:58 2022] POST /identity/v3/services => generated 251 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 22:26:59.004283 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f007a4c1-6709-4997-b21a-b8be26a42c3b None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:59.050935 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09af85d-af35-48d1-bd59-47f4caf7806e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:59.051571 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09af85d-af35-48d1-bd59-47f4caf7806e None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:59.051854 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09af85d-af35-48d1-bd59-47f4caf7806e None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:59.052123 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09af85d-af35-48d1-bd59-47f4caf7806e None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:59.052874 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09af85d-af35-48d1-bd59-47f4caf7806e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:59.053178 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09af85d-af35-48d1-bd59-47f4caf7806e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:59.054576 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09af85d-af35-48d1-bd59-47f4caf7806e None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:59.069952 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 436/877] 10.222.0.81 () {56 vars in 1115 bytes} [Wed Jul 20 22:26:58 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:59.072160 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4970f941-719c-40f0-815a-63ed1d59f882 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:59.199281 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4970f941-719c-40f0-815a-63ed1d59f882 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:59.230831 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad6e32e-eca5-44ba-b0a5-6fcba12f2419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:59.231689 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad6e32e-eca5-44ba-b0a5-6fcba12f2419 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:59.231976 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad6e32e-eca5-44ba-b0a5-6fcba12f2419 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:59.232257 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad6e32e-eca5-44ba-b0a5-6fcba12f2419 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:59.244369 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad6e32e-eca5-44ba-b0a5-6fcba12f2419 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:59.245812 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad6e32e-eca5-44ba-b0a5-6fcba12f2419 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:59.262220 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72a997e-a56f-4ba5-9546-32a6077a81ce None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:59.262538 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72a997e-a56f-4ba5-9546-32a6077a81ce None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:59.262782 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72a997e-a56f-4ba5-9546-32a6077a81ce None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:59.264244 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-6ad6e32e-eca5-44ba-b0a5-6fcba12f2419 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:26:59.264244 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:59.264244 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:59.264244 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:59.264244 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:59.264244 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:59.264244 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:59.264244 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:59.264244 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:59.264244 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:59.264244 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:59.264244 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:59.264244 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 22:26:59.264244 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 22:26:59.264244 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 22:26:59.265118 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 22:26:59.265118 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:59.265118 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:59.265118 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:59.265118 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:59.265118 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:59.265118 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:59.265118 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:59.265118 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:59.265118 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:59.265118 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:59.265118 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:59.265118 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:59.265118 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:59.265118 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:59.265118 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:59.265118 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:59.266013 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 22:26:59.266013 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 22:26:59.266013 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:26:59.266013 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:26:59.266249 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 437/878] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:26:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 22:26:59.266320 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 442/879] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Jul 20 22:26:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:59.271280 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad6e32e-eca5-44ba-b0a5-6fcba12f2419 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:59.271504 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557e4219-b780-4cdf-9680-ecdd959396e3 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:59.271842 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557e4219-b780-4cdf-9680-ecdd959396e3 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:59.272236 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557e4219-b780-4cdf-9680-ecdd959396e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:59.289176 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-557e4219-b780-4cdf-9680-ecdd959396e3 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:26:59.290382 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-557e4219-b780-4cdf-9680-ecdd959396e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:26:59.310913 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:59.346540 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:59.367288 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 438/880] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:26:59 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 22:26:59.386229 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557e4219-b780-4cdf-9680-ecdd959396e3 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:59.399839 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad6e32e-eca5-44ba-b0a5-6fcba12f2419 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:59.447916 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb64fd3-4b33-4bba-95b6-a554c438420a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:59.448937 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb64fd3-4b33-4bba-95b6-a554c438420a None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:59.449228 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb64fd3-4b33-4bba-95b6-a554c438420a None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:59.449514 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb64fd3-4b33-4bba-95b6-a554c438420a None admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:59.450251 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb64fd3-4b33-4bba-95b6-a554c438420a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:59.450568 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb64fd3-4b33-4bba-95b6-a554c438420a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:59.452506 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb64fd3-4b33-4bba-95b6-a554c438420a None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:59.472357 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 443/881] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Jul 20 22:26:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:59.480719 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb64fd3-4b33-4bba-95b6-a554c438420a None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:59.507337 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557e4219-b780-4cdf-9680-ecdd959396e3 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:59.550057 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d041f5-88af-4449-88e8-6b26b358244d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:59.550898 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d041f5-88af-4449-88e8-6b26b358244d None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:59.551217 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d041f5-88af-4449-88e8-6b26b358244d None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:59.551502 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d041f5-88af-4449-88e8-6b26b358244d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:59.552157 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d041f5-88af-4449-88e8-6b26b358244d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:59.553515 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d041f5-88af-4449-88e8-6b26b358244d None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:59.567622 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-81d041f5-88af-4449-88e8-6b26b358244d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 22:26:59.567622 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:59.567622 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 22:26:59.567622 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 22:26:59.567622 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 22:26:59.567622 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 22:26:59.567622 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 22:26:59.567622 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 22:26:59.567622 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 22:26:59.567622 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:26:59.567622 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 22:26:59.567622 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:26:59.567622 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:59.567622 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:59.567622 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:59.567622 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:59.567622 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:59.568582 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:59.568582 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:59.568582 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:59.568582 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:59.568582 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:59.568582 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:59.568582 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 22:26:59.568582 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 22:26:59.568582 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 22:26:59.568582 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 22:26:59.568582 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:59.568582 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:59.568582 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:59.568582 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:59.568582 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:59.568582 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:59.569412 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:59.569412 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:59.569412 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:59.569412 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:59.569412 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:59.569412 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:59.569412 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:59.569412 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:59.569412 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:59.569412 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:59.569412 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 22:26:59.569412 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 22:26:59.569412 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 22:26:59.569412 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:26:59.573591 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 439/882] 10.222.0.81 () {56 vars in 1126 bytes} [Wed Jul 20 22:26:59 2022] GET /identity/v3/services/object-store => generated 93 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:26:59.574833 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1154fed7-cad3-4289-a762-3b00f9c23729 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:59.575154 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1154fed7-cad3-4289-a762-3b00f9c23729 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:59.575438 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1154fed7-cad3-4289-a762-3b00f9c23729 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:59.579175 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 440/883] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:26:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:26:59.579175 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1154fed7-cad3-4289-a762-3b00f9c23729 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:59.589888 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:59.600750 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb64fd3-4b33-4bba-95b6-a554c438420a None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:59.635877 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:59.639756 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6b0acf-bbb3-4af8-bd3a-a7c1276f19f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:59.640804 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6b0acf-bbb3-4af8-bd3a-a7c1276f19f6 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:59.641087 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6b0acf-bbb3-4af8-bd3a-a7c1276f19f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:59.641364 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6b0acf-bbb3-4af8-bd3a-a7c1276f19f6 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:59.648951 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6b0acf-bbb3-4af8-bd3a-a7c1276f19f6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:59.650807 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6b0acf-bbb3-4af8-bd3a-a7c1276f19f6 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:59.668564 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-fc6b0acf-bbb3-4af8-bd3a-a7c1276f19f6 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 22:26:59.668564 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:26:59.668564 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:26:59.668564 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:26:59.668564 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:26:59.668564 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:26:59.668564 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:26:59.668564 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:26:59.668564 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:26:59.668564 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:26:59.668564 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:26:59.668564 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:26:59.668564 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 22:26:59.668564 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 22:26:59.668564 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 22:26:59.668564 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 22:26:59.669757 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:26:59.669757 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:26:59.669757 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 22:26:59.669757 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 22:26:59.669757 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 22:26:59.669757 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 22:26:59.669757 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:26:59.669757 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:26:59.669757 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:26:59.669757 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:26:59.669757 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:26:59.669757 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:26:59.669757 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:26:59.669757 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:26:59.669757 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:26:59.669757 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:26:59.669757 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:26:59.669757 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:26:59.670701 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:26:59.670701 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:26:59.670701 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 22:26:59.670701 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 22:26:59.670701 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 22:26:59.670701 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 22:26:59.670701 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 22:26:59.670701 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 22:26:59.670701 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 22:26:59.670701 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:26:59.670701 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 444/884] 10.222.0.81 () {56 vars in 1117 bytes} [Wed Jul 20 22:26:59 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 22:26:59.671750 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8741837-b261-45ea-b059-0f80152700d3 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:59.671750 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8741837-b261-45ea-b059-0f80152700d3 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:59.671750 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8741837-b261-45ea-b059-0f80152700d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:59.689135 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-f8741837-b261-45ea-b059-0f80152700d3 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:26:59.690467 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-f8741837-b261-45ea-b059-0f80152700d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:26:59.692302 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1154fed7-cad3-4289-a762-3b00f9c23729 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:59.727175 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc37f18-49d8-4591-81b6-cc3180651e61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:59.727851 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc37f18-49d8-4591-81b6-cc3180651e61 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:59.728141 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc37f18-49d8-4591-81b6-cc3180651e61 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:59.728421 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc37f18-49d8-4591-81b6-cc3180651e61 None admin] PATH_INFO: `/v3/services` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:59.729125 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc37f18-49d8-4591-81b6-cc3180651e61 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:59.729435 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc37f18-49d8-4591-81b6-cc3180651e61 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:59.730797 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc37f18-49d8-4591-81b6-cc3180651e61 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:59.740776 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 441/885] 10.222.0.81 () {56 vars in 1122 bytes} [Wed Jul 20 22:26:59 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:59.741651 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc37f18-49d8-4591-81b6-cc3180651e61 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:59.763425 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 445/886] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:26:59 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 22:26:59.764688 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8741837-b261-45ea-b059-0f80152700d3 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:59.856184 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc37f18-49d8-4591-81b6-cc3180651e61 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:59.860212 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:59.872258 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:26:59.881762 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8741837-b261-45ea-b059-0f80152700d3 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:26:59.894613 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a792b282-c800-4b24-ad2a-7699a2873e29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:59.895347 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a792b282-c800-4b24-ad2a-7699a2873e29 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:59.895635 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a792b282-c800-4b24-ad2a-7699a2873e29 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:59.896064 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a792b282-c800-4b24-ad2a-7699a2873e29 None admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:59.897263 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a792b282-c800-4b24-ad2a-7699a2873e29 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:59.897580 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a792b282-c800-4b24-ad2a-7699a2873e29 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:59.899504 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a792b282-c800-4b24-ad2a-7699a2873e29 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:59.918072 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 442/887] 10.222.0.81 () {56 vars in 1149 bytes} [Wed Jul 20 22:26:59 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:59.919124 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a792b282-c800-4b24-ad2a-7699a2873e29 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:26:59.923489 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b65a21-7693-41d5-9e6e-8d74ded8460c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:26:59.924321 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b65a21-7693-41d5-9e6e-8d74ded8460c None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:26:59.924635 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b65a21-7693-41d5-9e6e-8d74ded8460c None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:26:59.925032 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b65a21-7693-41d5-9e6e-8d74ded8460c None admin] PATH_INFO: `/v3/services` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:26:59.926417 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b65a21-7693-41d5-9e6e-8d74ded8460c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:26:59.926417 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b65a21-7693-41d5-9e6e-8d74ded8460c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:26:59.927468 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b65a21-7693-41d5-9e6e-8d74ded8460c None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:26:59.932416 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 446/888] 10.222.0.81 () {56 vars in 1122 bytes} [Wed Jul 20 22:26:59 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:26:59.933113 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b65a21-7693-41d5-9e6e-8d74ded8460c None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:00.056749 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a792b282-c800-4b24-ad2a-7699a2873e29 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:00.073704 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b65a21-7693-41d5-9e6e-8d74ded8460c None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:00.099038 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fed1261-8864-46cc-9ea5-887bb0c31623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:00.099716 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fed1261-8864-46cc-9ea5-887bb0c31623 None admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:00.100009 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fed1261-8864-46cc-9ea5-887bb0c31623 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:00.100289 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fed1261-8864-46cc-9ea5-887bb0c31623 None admin] PATH_INFO: `/v3/services` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:00.101016 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fed1261-8864-46cc-9ea5-887bb0c31623 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:00.102381 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fed1261-8864-46cc-9ea5-887bb0c31623 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:00.121670 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b470f6-e676-460a-92c0-15d21563f636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:00.122278 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b470f6-e676-460a-92c0-15d21563f636 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:00.122567 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b470f6-e676-460a-92c0-15d21563f636 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:00.122851 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b470f6-e676-460a-92c0-15d21563f636 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:00.140455 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:00.202946 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b470f6-e676-460a-92c0-15d21563f636 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:00.204578 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b470f6-e676-460a-92c0-15d21563f636 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:00.237157 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 443/889] 10.222.0.81 () {60 vars in 1141 bytes} [Wed Jul 20 22:26:59 2022] POST /identity/v3/services => generated 269 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:27:00.325178 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 447/890] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Jul 20 22:26:59 2022] GET /identity/v3/auth/tokens => generated 1252 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 22:27:00.331542 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fed1261-8864-46cc-9ea5-887bb0c31623 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:00.368444 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:00.446430 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fed1261-8864-46cc-9ea5-887bb0c31623 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:00.482313 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e0dd55-b8dd-44b0-be4d-056d1be11a8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:00.483162 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e0dd55-b8dd-44b0-be4d-056d1be11a8f None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:00.483449 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e0dd55-b8dd-44b0-be4d-056d1be11a8f None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:00.483726 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e0dd55-b8dd-44b0-be4d-056d1be11a8f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:00.494810 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e0dd55-b8dd-44b0-be4d-056d1be11a8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:00.496299 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e0dd55-b8dd-44b0-be4d-056d1be11a8f None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:00.514397 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-00e0dd55-b8dd-44b0-be4d-056d1be11a8f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:27:00.514397 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:27:00.514397 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:27:00.514397 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:27:00.514397 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:27:00.514397 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:27:00.514397 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:27:00.514397 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:27:00.514397 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:27:00.514397 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:27:00.514397 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:27:00.514397 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:27:00.514397 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 22:27:00.514397 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 22:27:00.514397 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 22:27:00.515386 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 22:27:00.515386 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:27:00.515386 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:27:00.515386 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:27:00.515386 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:27:00.515386 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:27:00.515386 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:27:00.515386 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:27:00.515386 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:27:00.515386 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:27:00.515386 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:27:00.515386 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:27:00.515386 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:27:00.515386 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:27:00.515386 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:27:00.515386 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:27:00.515386 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:27:00.516719 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 22:27:00.516719 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 22:27:00.516719 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:27:00.516719 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:27:00.516719 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 444/891] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Jul 20 22:27:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:27:00.522188 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b470f6-e676-460a-92c0-15d21563f636 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:00.573565 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:00.629957 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b470f6-e676-460a-92c0-15d21563f636 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:00.666105 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abd9c94-3d5b-48cd-acfe-457fd91f2ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:00.666777 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abd9c94-3d5b-48cd-acfe-457fd91f2ba5 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:00.667091 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abd9c94-3d5b-48cd-acfe-457fd91f2ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:00.667378 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abd9c94-3d5b-48cd-acfe-457fd91f2ba5 None admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:00.668111 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abd9c94-3d5b-48cd-acfe-457fd91f2ba5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:27:00.668417 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abd9c94-3d5b-48cd-acfe-457fd91f2ba5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:00.669795 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abd9c94-3d5b-48cd-acfe-457fd91f2ba5 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:00.682748 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 448/892] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Jul 20 22:27:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:27:00.687922 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e0dd55-b8dd-44b0-be4d-056d1be11a8f None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:00.773245 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:00.793413 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e0dd55-b8dd-44b0-be4d-056d1be11a8f None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:00.830251 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901663af-d4aa-4e05-935c-b8ccf28679eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:00.831100 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901663af-d4aa-4e05-935c-b8ccf28679eb None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:00.831390 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901663af-d4aa-4e05-935c-b8ccf28679eb None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:00.831669 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901663af-d4aa-4e05-935c-b8ccf28679eb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:00.838568 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901663af-d4aa-4e05-935c-b8ccf28679eb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:00.840030 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901663af-d4aa-4e05-935c-b8ccf28679eb None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:00.855661 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-901663af-d4aa-4e05-935c-b8ccf28679eb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 22:27:00.855661 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:27:00.855661 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:27:00.855661 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:27:00.855661 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:27:00.855661 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:27:00.855661 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:27:00.855661 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:27:00.855661 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:27:00.855661 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:27:00.855661 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:27:00.855661 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:27:00.855661 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 22:27:00.855661 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 22:27:00.855661 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 22:27:00.856577 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 22:27:00.856577 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:27:00.856577 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:27:00.856577 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:27:00.856577 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:27:00.856577 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:27:00.856577 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:27:00.856577 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:27:00.856577 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:27:00.856577 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:27:00.856577 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:27:00.856577 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:27:00.856577 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:27:00.856577 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:27:00.856577 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:27:00.856577 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:27:00.856577 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:27:00.857868 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 22:27:00.857868 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 22:27:00.857868 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 22:27:00.857868 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 22:27:00.857868 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 22:27:00.857868 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 22:27:00.857868 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 22:27:00.857868 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:27:00.857868 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 445/893] 10.222.0.81 () {56 vars in 1111 bytes} [Wed Jul 20 22:27:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:27:00.863742 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abd9c94-3d5b-48cd-acfe-457fd91f2ba5 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:00.971618 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abd9c94-3d5b-48cd-acfe-457fd91f2ba5 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:00.973984 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:01.006792 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fccfb77-392f-41ca-bf15-82d3e240baea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:01.007545 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fccfb77-392f-41ca-bf15-82d3e240baea None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:01.007839 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fccfb77-392f-41ca-bf15-82d3e240baea None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:01.008121 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fccfb77-392f-41ca-bf15-82d3e240baea None admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:01.008912 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fccfb77-392f-41ca-bf15-82d3e240baea None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:27:01.009223 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fccfb77-392f-41ca-bf15-82d3e240baea None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:01.010660 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fccfb77-392f-41ca-bf15-82d3e240baea None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:01.023057 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 449/894] 10.222.0.81 () {56 vars in 1148 bytes} [Wed Jul 20 22:27:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:27:01.028570 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901663af-d4aa-4e05-935c-b8ccf28679eb None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:01.130824 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5977ab-551c-468a-b36c-e5251cfbef5d None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:01.130824 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5977ab-551c-468a-b36c-e5251cfbef5d None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:01.131159 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5977ab-551c-468a-b36c-e5251cfbef5d None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:01.132734 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 450/895] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:27:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:27:01.137594 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baad5379-2db2-4974-9784-dd2556cd6278 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:01.137882 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baad5379-2db2-4974-9784-dd2556cd6278 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:01.138388 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baad5379-2db2-4974-9784-dd2556cd6278 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:01.141728 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901663af-d4aa-4e05-935c-b8ccf28679eb None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:01.155784 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-baad5379-2db2-4974-9784-dd2556cd6278 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:27:01.156995 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-baad5379-2db2-4974-9784-dd2556cd6278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:27:01.179964 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db605e21-8390-4348-a328-d79b6479df61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:01.180986 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db605e21-8390-4348-a328-d79b6479df61 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:01.181480 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db605e21-8390-4348-a328-d79b6479df61 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:01.181968 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db605e21-8390-4348-a328-d79b6479df61 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:01.182997 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db605e21-8390-4348-a328-d79b6479df61 None admin] RBAC: Adding query filter params (role.id=a3e411d61a234343b961e8bf78a55a11, scope.project.id=385edf2e59e546d090ace36600e29032, user.id=90915e1264324f16bc7bae7e46e0c97b) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:27:01.183561 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db605e21-8390-4348-a328-d79b6479df61 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:01.185235 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db605e21-8390-4348-a328-d79b6479df61 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:01.192289 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 446/896] 10.222.0.81 () {56 vars in 1374 bytes} [Wed Jul 20 22:27:01 2022] GET /identity/v3/role_assignments?user.id=90915e1264324f16bc7bae7e46e0c97b&scope.project.id=385edf2e59e546d090ace36600e29032&role.id=a3e411d61a234343b961e8bf78a55a11 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:27:01.241039 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 451/897] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:27:01 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 22:27:01.264762 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db605e21-8390-4348-a328-d79b6479df61 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:01.274815 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:01.374837 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db605e21-8390-4348-a328-d79b6479df61 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:01.409528 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796e1bb0-cb19-4a19-857c-4221e766e8a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:01.410392 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796e1bb0-cb19-4a19-857c-4221e766e8a5 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:01.410674 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796e1bb0-cb19-4a19-857c-4221e766e8a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:01.410951 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796e1bb0-cb19-4a19-857c-4221e766e8a5 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:01.411884 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796e1bb0-cb19-4a19-857c-4221e766e8a5 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:01.413243 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796e1bb0-cb19-4a19-857c-4221e766e8a5 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:01.425977 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-796e1bb0-cb19-4a19-857c-4221e766e8a5 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 20 22:27:01.425977 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:27:01.425977 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 22:27:01.425977 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 22:27:01.425977 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 22:27:01.425977 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 22:27:01.425977 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 22:27:01.425977 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 22:27:01.425977 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 20 22:27:01.425977 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:27:01.425977 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 22:27:01.425977 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:27:01.425977 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:27:01.425977 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:27:01.425977 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:27:01.425977 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:27:01.425977 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:27:01.426967 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:27:01.426967 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:27:01.426967 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:27:01.426967 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:27:01.426967 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:27:01.426967 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:27:01.426967 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 22:27:01.426967 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 22:27:01.426967 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 22:27:01.426967 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 22:27:01.426967 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:27:01.426967 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:27:01.426967 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:27:01.426967 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:27:01.426967 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:27:01.426967 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:27:01.427823 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:27:01.427823 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:27:01.427823 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:27:01.427823 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:27:01.427823 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:27:01.427823 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:27:01.427823 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:27:01.427823 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:27:01.427823 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:27:01.427823 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:27:01.427823 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 22:27:01.427823 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 22:27:01.427823 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 20 22:27:01.427823 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:27:01.427823 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 447/898] 10.222.0.81 () {56 vars in 1114 bytes} [Wed Jul 20 22:27:01 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:27:01.433440 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baad5379-2db2-4974-9784-dd2556cd6278 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:01.532867 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:01.539872 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baad5379-2db2-4974-9784-dd2556cd6278 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:01.576117 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0748da6-c8c6-4367-8f5d-7a8074543e0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:01.576792 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0748da6-c8c6-4367-8f5d-7a8074543e0b None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:01.577079 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0748da6-c8c6-4367-8f5d-7a8074543e0b None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:01.577360 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0748da6-c8c6-4367-8f5d-7a8074543e0b None admin] PATH_INFO: `/v3/services` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:01.578068 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0748da6-c8c6-4367-8f5d-7a8074543e0b None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:27:01.578373 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0748da6-c8c6-4367-8f5d-7a8074543e0b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:01.579817 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0748da6-c8c6-4367-8f5d-7a8074543e0b None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:01.585002 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 452/899] 10.222.0.81 () {56 vars in 1114 bytes} [Wed Jul 20 22:27:01 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:27:01.590564 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796e1bb0-cb19-4a19-857c-4221e766e8a5 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:01.665954 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:01.698493 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796e1bb0-cb19-4a19-857c-4221e766e8a5 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:01.733315 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82562c18-db1a-48a1-9b27-d601c0557158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:01.734001 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82562c18-db1a-48a1-9b27-d601c0557158 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:01.734293 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82562c18-db1a-48a1-9b27-d601c0557158 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:01.734584 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82562c18-db1a-48a1-9b27-d601c0557158 None admin] PATH_INFO: `/v3/services` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:01.735326 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82562c18-db1a-48a1-9b27-d601c0557158 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:27:01.736200 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82562c18-db1a-48a1-9b27-d601c0557158 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:01.737574 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82562c18-db1a-48a1-9b27-d601c0557158 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:01.742932 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 448/900] 10.222.0.81 () {56 vars in 1114 bytes} [Wed Jul 20 22:27:01 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:27:01.748356 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0748da6-c8c6-4367-8f5d-7a8074543e0b None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:01.857471 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0748da6-c8c6-4367-8f5d-7a8074543e0b None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:01.893167 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71745075-4860-4e34-adb1-9d00efbd15c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:01.893862 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71745075-4860-4e34-adb1-9d00efbd15c9 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:01.894148 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71745075-4860-4e34-adb1-9d00efbd15c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:01.894429 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71745075-4860-4e34-adb1-9d00efbd15c9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:01.895249 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71745075-4860-4e34-adb1-9d00efbd15c9 None admin] RBAC: Adding query filter params (interface=public, service_id=859b63b373024f7288d3758f87486680, region_id=RegionOne) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:27:01.895566 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71745075-4860-4e34-adb1-9d00efbd15c9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:01.896946 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71745075-4860-4e34-adb1-9d00efbd15c9 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:01.902747 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 453/901] 10.222.0.81 () {56 vars in 1251 bytes} [Wed Jul 20 22:27:01 2022] GET /identity/v3/endpoints?service_id=859b63b373024f7288d3758f87486680&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:27:01.908806 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82562c18-db1a-48a1-9b27-d601c0557158 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:02.016275 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82562c18-db1a-48a1-9b27-d601c0557158 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:02.052389 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eae5fd5-aa1b-49c9-8874-c61fe3365c35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:02.053122 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eae5fd5-aa1b-49c9-8874-c61fe3365c35 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:02.053415 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eae5fd5-aa1b-49c9-8874-c61fe3365c35 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:02.053702 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eae5fd5-aa1b-49c9-8874-c61fe3365c35 None admin] PATH_INFO: `/v3/services` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:02.054398 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eae5fd5-aa1b-49c9-8874-c61fe3365c35 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:02.055878 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eae5fd5-aa1b-49c9-8874-c61fe3365c35 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:02.057047 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:02.062466 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 449/902] 10.222.0.81 () {56 vars in 1087 bytes} [Wed Jul 20 22:27:01 2022] GET /identity/v3/services => generated 1268 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:27:02.160518 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799bf53b-0f04-4222-944a-d2a465f0604d None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:02.160886 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799bf53b-0f04-4222-944a-d2a465f0604d None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:02.161116 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799bf53b-0f04-4222-944a-d2a465f0604d None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:02.163204 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 454/903] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:27:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:27:02.168898 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6efcf0-8beb-42dc-a8f0-fa7a7a8b5897 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:02.169189 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6efcf0-8beb-42dc-a8f0-fa7a7a8b5897 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:02.169467 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6efcf0-8beb-42dc-a8f0-fa7a7a8b5897 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:02.196239 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-5a6efcf0-8beb-42dc-a8f0-fa7a7a8b5897 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:27:02.197732 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-5a6efcf0-8beb-42dc-a8f0-fa7a7a8b5897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:27:02.277213 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 450/904] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:27:02 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 22:27:02.300038 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799bf53b-0f04-4222-944a-d2a465f0604d None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:02.326663 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:02.409243 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799bf53b-0f04-4222-944a-d2a465f0604d None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:02.449281 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a44cf9-a833-4489-90e4-41858b728945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:02.449959 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a44cf9-a833-4489-90e4-41858b728945 None admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:02.450247 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a44cf9-a833-4489-90e4-41858b728945 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:02.450529 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a44cf9-a833-4489-90e4-41858b728945 None admin] PATH_INFO: `/v3/services` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:02.451302 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a44cf9-a833-4489-90e4-41858b728945 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:02.452677 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a44cf9-a833-4489-90e4-41858b728945 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:02.479545 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4ecb00-bfe6-4ccd-84cb-3b47134022a9 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:02.479841 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4ecb00-bfe6-4ccd-84cb-3b47134022a9 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:02.480124 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4ecb00-bfe6-4ccd-84cb-3b47134022a9 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:02.481627 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 451/905] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:27:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 22:27:02.486268 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e649ac4-cb53-408b-8bac-09fb418f68a7 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:02.487204 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e649ac4-cb53-408b-8bac-09fb418f68a7 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:02.487392 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e649ac4-cb53-408b-8bac-09fb418f68a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:02.503915 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-2e649ac4-cb53-408b-8bac-09fb418f68a7 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:27:02.505098 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-2e649ac4-cb53-408b-8bac-09fb418f68a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:27:02.525403 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 455/906] 10.222.0.81 () {60 vars in 1141 bytes} [Wed Jul 20 22:27:02 2022] POST /identity/v3/services => generated 241 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:27:02.597531 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 452/907] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:27:02 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 22:27:02.620285 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a44cf9-a833-4489-90e4-41858b728945 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:02.712401 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:02.733430 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a44cf9-a833-4489-90e4-41858b728945 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:02.769720 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7594a891-aee1-4f2f-8a6c-b5d8df6f3b63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:02.770565 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7594a891-aee1-4f2f-8a6c-b5d8df6f3b63 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:02.770853 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7594a891-aee1-4f2f-8a6c-b5d8df6f3b63 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:02.771158 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7594a891-aee1-4f2f-8a6c-b5d8df6f3b63 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:02.771824 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7594a891-aee1-4f2f-8a6c-b5d8df6f3b63 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:02.773211 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7594a891-aee1-4f2f-8a6c-b5d8df6f3b63 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:02.787240 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-7594a891-aee1-4f2f-8a6c-b5d8df6f3b63 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 20 22:27:02.787240 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:27:02.787240 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 22:27:02.787240 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 22:27:02.787240 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 22:27:02.787240 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 22:27:02.787240 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 22:27:02.787240 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 22:27:02.787240 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 20 22:27:02.787240 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:27:02.787240 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 22:27:02.787240 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:27:02.787240 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:27:02.787240 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:27:02.787240 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:27:02.787240 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:27:02.788208 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:27:02.788208 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:27:02.788208 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:27:02.788208 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:27:02.788208 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:27:02.788208 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:27:02.788208 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:27:02.788208 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 22:27:02.788208 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 22:27:02.788208 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 22:27:02.788208 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 22:27:02.788208 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:27:02.788208 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:27:02.788208 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:27:02.788208 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:27:02.788208 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:27:02.789045 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:27:02.789045 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:27:02.789045 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:27:02.789045 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:27:02.789045 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:27:02.789045 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:27:02.789045 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:27:02.789045 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:27:02.789045 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:27:02.789045 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:27:02.789045 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:27:02.789045 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 22:27:02.789045 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 22:27:02.789045 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 20 22:27:02.789045 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:27:02.789045 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 456/908] 10.222.0.81 () {56 vars in 1132 bytes} [Wed Jul 20 22:27:02 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:27:02.795992 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e649ac4-cb53-408b-8bac-09fb418f68a7 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:02.855057 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:02.904841 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e649ac4-cb53-408b-8bac-09fb418f68a7 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:02.942194 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9728853-5635-4c00-affa-67b798133ced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:02.942925 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9728853-5635-4c00-affa-67b798133ced None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:02.943248 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9728853-5635-4c00-affa-67b798133ced None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:02.943531 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9728853-5635-4c00-affa-67b798133ced None admin] PATH_INFO: `/v3/services` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:02.944246 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9728853-5635-4c00-affa-67b798133ced None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:27:02.944555 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9728853-5635-4c00-affa-67b798133ced None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:02.945919 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9728853-5635-4c00-affa-67b798133ced None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:02.951203 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 453/909] 10.222.0.81 () {56 vars in 1126 bytes} [Wed Jul 20 22:27:02 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:27:02.960086 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7594a891-aee1-4f2f-8a6c-b5d8df6f3b63 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:03.063598 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7594a891-aee1-4f2f-8a6c-b5d8df6f3b63 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:03.098767 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd2148d-5156-4393-9eb6-374d42f843f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:03.099978 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd2148d-5156-4393-9eb6-374d42f843f0 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:03.100515 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd2148d-5156-4393-9eb6-374d42f843f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:03.100995 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd2148d-5156-4393-9eb6-374d42f843f0 None admin] PATH_INFO: `/v3/services` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:03.101899 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd2148d-5156-4393-9eb6-374d42f843f0 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:27:03.102408 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd2148d-5156-4393-9eb6-374d42f843f0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:03.104103 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd2148d-5156-4393-9eb6-374d42f843f0 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:03.109705 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 457/910] 10.222.0.81 () {56 vars in 1126 bytes} [Wed Jul 20 22:27:02 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:27:03.116340 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9728853-5635-4c00-affa-67b798133ced None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:03.226230 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9728853-5635-4c00-affa-67b798133ced None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:03.245378 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:03.263621 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef050e4-a215-4fe0-9fb6-d9ea7a6f49da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:03.264357 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef050e4-a215-4fe0-9fb6-d9ea7a6f49da None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:03.264641 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef050e4-a215-4fe0-9fb6-d9ea7a6f49da None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:03.264973 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef050e4-a215-4fe0-9fb6-d9ea7a6f49da None admin] PATH_INFO: `/v3/endpoints` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:03.265766 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef050e4-a215-4fe0-9fb6-d9ea7a6f49da None admin] RBAC: Adding query filter params (interface=public, service_id=c198b399e0f04902b55b11a3841ae419, region_id=RegionOne) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:27:03.266081 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef050e4-a215-4fe0-9fb6-d9ea7a6f49da None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:03.268026 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef050e4-a215-4fe0-9fb6-d9ea7a6f49da None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:03.276317 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 454/911] 10.222.0.81 () {56 vars in 1251 bytes} [Wed Jul 20 22:27:03 2022] GET /identity/v3/endpoints?service_id=c198b399e0f04902b55b11a3841ae419&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:27:03.281804 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd2148d-5156-4393-9eb6-374d42f843f0 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:03.295405 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:03.396232 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd2148d-5156-4393-9eb6-374d42f843f0 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:03.422769 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1d84aa-19c3-427c-9e57-67558a0df463 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:03.423309 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1d84aa-19c3-427c-9e57-67558a0df463 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:03.423640 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1d84aa-19c3-427c-9e57-67558a0df463 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:03.425363 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 455/912] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:27:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 22:27:03.430228 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aab46e-790f-4185-bfa2-c730c1512423 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:03.430643 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aab46e-790f-4185-bfa2-c730c1512423 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:03.431043 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aab46e-790f-4185-bfa2-c730c1512423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:03.433096 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e47123-a212-41cc-a7e6-178f881ae650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:03.433934 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e47123-a212-41cc-a7e6-178f881ae650 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:03.434446 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e47123-a212-41cc-a7e6-178f881ae650 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:03.434874 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e47123-a212-41cc-a7e6-178f881ae650 None admin] PATH_INFO: `/v3/services` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:03.435728 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e47123-a212-41cc-a7e6-178f881ae650 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:03.438259 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e47123-a212-41cc-a7e6-178f881ae650 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:03.446217 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 458/913] 10.222.0.81 () {56 vars in 1087 bytes} [Wed Jul 20 22:27:03 2022] GET /identity/v3/services => generated 1497 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:27:03.452210 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-28aab46e-790f-4185-bfa2-c730c1512423 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:27:03.452210 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-28aab46e-790f-4185-bfa2-c730c1512423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:27:03.528816 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 456/914] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:27:03 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 22:27:03.552733 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e47123-a212-41cc-a7e6-178f881ae650 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:03.641069 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:03.665941 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e47123-a212-41cc-a7e6-178f881ae650 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:03.707417 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec62bde-7d04-4c01-9206-14a645706f24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:03.708259 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec62bde-7d04-4c01-9206-14a645706f24 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:03.708549 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec62bde-7d04-4c01-9206-14a645706f24 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:03.708830 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec62bde-7d04-4c01-9206-14a645706f24 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:03.719431 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec62bde-7d04-4c01-9206-14a645706f24 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 22:27:03.728257 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-5ec62bde-7d04-4c01-9206-14a645706f24 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 22:27:03.728257 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:27:03.728257 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:27:03.728257 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:27:03.728257 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:27:03.728257 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:27:03.728257 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:27:03.728257 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:27:03.728257 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:27:03.728257 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:27:03.728257 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:27:03.728257 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:27:03.728257 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 22:27:03.728257 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 22:27:03.728257 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 22:27:03.728257 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 22:27:03.729175 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 22:27:03.729175 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 22:27:03.729175 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 22:27:03.729175 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 22:27:03.729175 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:27:03.729175 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:27:03.729175 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:27:03.729175 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:27:03.729175 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:27:03.729175 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:27:03.729175 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:27:03.729175 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:27:03.729175 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:27:03.729175 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:27:03.729175 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:27:03.729175 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:27:03.730004 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:27:03.730004 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:27:03.730004 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:27:03.730004 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:27:03.730004 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 22:27:03.730004 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 22:27:03.730004 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 22:27:03.730004 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 22:27:03.730004 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 22:27:03.730004 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 22:27:03.730004 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 22:27:03.730004 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:27:03.730004 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 459/915] 10.222.0.81 () {56 vars in 1120 bytes} [Wed Jul 20 22:27:03 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:27:03.737708 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aab46e-790f-4185-bfa2-c730c1512423 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:03.850694 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aab46e-790f-4185-bfa2-c730c1512423 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:03.865266 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:03.888058 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bfdcc1-a0bc-468a-8c3c-9ce206e234fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:03.888746 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bfdcc1-a0bc-468a-8c3c-9ce206e234fc None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:03.889037 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bfdcc1-a0bc-468a-8c3c-9ce206e234fc None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:03.889323 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bfdcc1-a0bc-468a-8c3c-9ce206e234fc None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:03.890077 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bfdcc1-a0bc-468a-8c3c-9ce206e234fc None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:27:03.890391 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bfdcc1-a0bc-468a-8c3c-9ce206e234fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:03.891930 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bfdcc1-a0bc-468a-8c3c-9ce206e234fc None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:03.900719 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 457/916] 10.222.0.81 () {56 vars in 1115 bytes} [Wed Jul 20 22:27:03 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:27:03.906549 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec62bde-7d04-4c01-9206-14a645706f24 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:04.024026 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec62bde-7d04-4c01-9206-14a645706f24 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:04.067851 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a419e62-8229-4b61-bbae-aee8cb98d018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:04.068689 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a419e62-8229-4b61-bbae-aee8cb98d018 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:04.068969 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a419e62-8229-4b61-bbae-aee8cb98d018 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:04.069272 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a419e62-8229-4b61-bbae-aee8cb98d018 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:04.071240 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:04.080589 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a419e62-8229-4b61-bbae-aee8cb98d018 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:04.082253 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a419e62-8229-4b61-bbae-aee8cb98d018 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:04.100762 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-6a419e62-8229-4b61-bbae-aee8cb98d018 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:27:04.100762 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:27:04.100762 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:27:04.100762 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:27:04.100762 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:27:04.100762 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:27:04.100762 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:27:04.100762 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:27:04.100762 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:27:04.100762 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:27:04.100762 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:27:04.100762 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:27:04.100762 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 22:27:04.100762 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 22:27:04.100762 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 22:27:04.101656 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 22:27:04.101656 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:27:04.101656 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:27:04.101656 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:27:04.101656 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:27:04.101656 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:27:04.101656 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:27:04.101656 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:27:04.101656 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:27:04.101656 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:27:04.101656 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:27:04.101656 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:27:04.101656 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:27:04.101656 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:27:04.101656 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:27:04.101656 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:27:04.101656 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:27:04.102535 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 22:27:04.102535 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 22:27:04.102535 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:27:04.102535 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:27:04.102535 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 460/917] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Jul 20 22:27:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:27:04.108969 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bfdcc1-a0bc-468a-8c3c-9ce206e234fc None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:04.231658 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bfdcc1-a0bc-468a-8c3c-9ce206e234fc None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:04.256851 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f856692-5bbf-4d10-96ad-d4e105650b49 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:04.257154 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f856692-5bbf-4d10-96ad-d4e105650b49 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:04.257420 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f856692-5bbf-4d10-96ad-d4e105650b49 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:04.258946 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 461/918] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:27:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:27:04.263919 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ff0dd4-aa7a-4165-ae32-3c34e79c310f None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:04.264204 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ff0dd4-aa7a-4165-ae32-3c34e79c310f None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:04.264482 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ff0dd4-aa7a-4165-ae32-3c34e79c310f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:04.270896 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad967ecd-77ed-42ed-926e-bbfd6a15ebe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:04.271947 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad967ecd-77ed-42ed-926e-bbfd6a15ebe5 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:04.272440 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad967ecd-77ed-42ed-926e-bbfd6a15ebe5 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:04.272936 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad967ecd-77ed-42ed-926e-bbfd6a15ebe5 None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:04.273865 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad967ecd-77ed-42ed-926e-bbfd6a15ebe5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:27:04.274460 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad967ecd-77ed-42ed-926e-bbfd6a15ebe5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:04.276082 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad967ecd-77ed-42ed-926e-bbfd6a15ebe5 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:04.283337 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-36ff0dd4-aa7a-4165-ae32-3c34e79c310f None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:27:04.284801 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-36ff0dd4-aa7a-4165-ae32-3c34e79c310f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:27:04.290487 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 458/919] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Jul 20 22:27:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:27:04.297367 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad967ecd-77ed-42ed-926e-bbfd6a15ebe5 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:04.335970 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:04.379049 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 462/920] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:27:04 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 22:27:04.401828 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ff0dd4-aa7a-4165-ae32-3c34e79c310f None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:04.423503 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad967ecd-77ed-42ed-926e-bbfd6a15ebe5 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:04.467189 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80e02c9-36c9-403f-a6d5-654a1209eea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:04.468225 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80e02c9-36c9-403f-a6d5-654a1209eea5 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:04.468609 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80e02c9-36c9-403f-a6d5-654a1209eea5 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:04.468986 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80e02c9-36c9-403f-a6d5-654a1209eea5 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:04.476795 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80e02c9-36c9-403f-a6d5-654a1209eea5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:04.478309 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80e02c9-36c9-403f-a6d5-654a1209eea5 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:04.496488 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-c80e02c9-36c9-403f-a6d5-654a1209eea5 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 22:27:04.496488 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:27:04.496488 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:27:04.496488 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:27:04.496488 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:27:04.496488 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:27:04.496488 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:27:04.496488 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:27:04.496488 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:27:04.496488 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:27:04.496488 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:27:04.496488 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:27:04.496488 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 22:27:04.496488 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 22:27:04.496488 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 22:27:04.496488 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 22:27:04.497708 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:27:04.497708 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:27:04.497708 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 22:27:04.497708 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 22:27:04.497708 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 22:27:04.497708 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 22:27:04.497708 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:27:04.497708 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:27:04.497708 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:27:04.497708 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:27:04.497708 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:27:04.497708 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:27:04.497708 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:27:04.497708 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:27:04.497708 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:27:04.497708 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:27:04.497708 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:27:04.497708 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:27:04.498875 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:27:04.498875 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:27:04.498875 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 22:27:04.498875 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 22:27:04.498875 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 22:27:04.498875 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 22:27:04.498875 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 22:27:04.498875 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 22:27:04.498875 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 22:27:04.498875 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:27:04.505884 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 459/921] 10.222.0.81 () {56 vars in 1117 bytes} [Wed Jul 20 22:27:04 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 22:27:04.507980 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80e02c9-36c9-403f-a6d5-654a1209eea5 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:04.544784 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ff0dd4-aa7a-4165-ae32-3c34e79c310f None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:04.567710 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:04.593320 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c649a2fe-845e-4d0d-8174-32714dc3ef27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:04.595872 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c649a2fe-845e-4d0d-8174-32714dc3ef27 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:04.596414 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c649a2fe-845e-4d0d-8174-32714dc3ef27 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:04.597190 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c649a2fe-845e-4d0d-8174-32714dc3ef27 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:04.598589 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c649a2fe-845e-4d0d-8174-32714dc3ef27 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:04.600726 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c649a2fe-845e-4d0d-8174-32714dc3ef27 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:04.610085 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:04.617129 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-c649a2fe-845e-4d0d-8174-32714dc3ef27 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 20 22:27:04.617129 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:27:04.617129 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 22:27:04.617129 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 22:27:04.617129 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 22:27:04.617129 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 22:27:04.617129 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 22:27:04.617129 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 22:27:04.617129 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 20 22:27:04.617129 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:27:04.617129 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 22:27:04.617129 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:27:04.617129 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:27:04.617129 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:27:04.617129 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:27:04.617129 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:27:04.617129 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:27:04.621487 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:27:04.621487 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:27:04.621487 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:27:04.621487 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:27:04.621487 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:27:04.621487 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:27:04.621487 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 22:27:04.621487 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 22:27:04.621487 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 22:27:04.621487 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 22:27:04.621487 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:27:04.621487 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:27:04.621487 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:27:04.621487 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:27:04.621487 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:27:04.621487 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:27:04.622306 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:27:04.622306 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:27:04.622306 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:27:04.622306 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:27:04.622306 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:27:04.622306 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:27:04.622306 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:27:04.622306 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:27:04.622306 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:27:04.622306 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:27:04.622306 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 22:27:04.622306 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 22:27:04.622306 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 20 22:27:04.622306 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:27:04.622306 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 463/922] 10.222.0.81 () {56 vars in 1114 bytes} [Wed Jul 20 22:27:04 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 22:27:04.623117 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80e02c9-36c9-403f-a6d5-654a1209eea5 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:04.624897 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c649a2fe-845e-4d0d-8174-32714dc3ef27 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:04.660782 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0c3a3c-c98b-4a37-b35a-ff7062a12032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:04.661849 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0c3a3c-c98b-4a37-b35a-ff7062a12032 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:04.662146 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0c3a3c-c98b-4a37-b35a-ff7062a12032 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:04.662428 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0c3a3c-c98b-4a37-b35a-ff7062a12032 None admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:04.663349 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0c3a3c-c98b-4a37-b35a-ff7062a12032 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:27:04.663669 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0c3a3c-c98b-4a37-b35a-ff7062a12032 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:04.665269 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0c3a3c-c98b-4a37-b35a-ff7062a12032 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:04.689423 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 460/923] 10.222.0.81 () {56 vars in 1149 bytes} [Wed Jul 20 22:27:04 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 22:27:04.690351 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0c3a3c-c98b-4a37-b35a-ff7062a12032 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:04.739407 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c649a2fe-845e-4d0d-8174-32714dc3ef27 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:04.776733 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6710f67c-f8db-40ca-8b86-2619edb99ef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:04.778012 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6710f67c-f8db-40ca-8b86-2619edb99ef7 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:04.778545 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6710f67c-f8db-40ca-8b86-2619edb99ef7 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:04.779041 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6710f67c-f8db-40ca-8b86-2619edb99ef7 None admin] PATH_INFO: `/v3/services` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:04.779987 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6710f67c-f8db-40ca-8b86-2619edb99ef7 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:27:04.780492 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6710f67c-f8db-40ca-8b86-2619edb99ef7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:04.782207 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6710f67c-f8db-40ca-8b86-2619edb99ef7 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:04.788158 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 464/924] 10.222.0.81 () {56 vars in 1114 bytes} [Wed Jul 20 22:27:04 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:27:04.789741 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4647c520-cd41-4c30-9970-2f4a3cde6bea None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:04.790264 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4647c520-cd41-4c30-9970-2f4a3cde6bea None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:04.790790 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4647c520-cd41-4c30-9970-2f4a3cde6bea None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:04.792568 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 465/925] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:27:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:27:04.793493 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4647c520-cd41-4c30-9970-2f4a3cde6bea None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:04.794332 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0c3a3c-c98b-4a37-b35a-ff7062a12032 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:04.831418 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0220c71b-ec19-441d-b549-cba28d326015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:04.832238 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0220c71b-ec19-441d-b549-cba28d326015 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:04.832520 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0220c71b-ec19-441d-b549-cba28d326015 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:04.832799 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0220c71b-ec19-441d-b549-cba28d326015 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:04.842984 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0220c71b-ec19-441d-b549-cba28d326015 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:04.844840 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0220c71b-ec19-441d-b549-cba28d326015 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:04.848378 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:04.863687 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-0220c71b-ec19-441d-b549-cba28d326015 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:27:04.863687 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:27:04.863687 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:27:04.863687 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:27:04.863687 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:27:04.863687 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:27:04.863687 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:27:04.863687 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:27:04.863687 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:27:04.863687 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:27:04.863687 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:27:04.863687 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:27:04.863687 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 22:27:04.863687 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 22:27:04.863687 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 22:27:04.865030 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 22:27:04.865030 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:27:04.865030 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:27:04.865030 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:27:04.865030 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:27:04.865030 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:27:04.865030 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:27:04.865030 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:27:04.865030 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:27:04.865030 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:27:04.865030 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:27:04.865030 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:27:04.865030 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:27:04.865030 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:27:04.865030 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:27:04.865030 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:27:04.865030 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:27:04.865910 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 22:27:04.865910 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 22:27:04.865910 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:27:04.865910 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:27:04.865910 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 461/926] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Jul 20 22:27:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:27:04.866290 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f388e2f-079a-447a-810b-31402e9f96ca None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:04.866513 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f388e2f-079a-447a-810b-31402e9f96ca None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:04.866782 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f388e2f-079a-447a-810b-31402e9f96ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:04.873488 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:04.884448 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-8f388e2f-079a-447a-810b-31402e9f96ca None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:27:04.885662 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-8f388e2f-079a-447a-810b-31402e9f96ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:27:04.912857 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4647c520-cd41-4c30-9970-2f4a3cde6bea None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:04.947174 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799dff3f-3a7a-498d-a939-a1c3155e243e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:04.947834 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799dff3f-3a7a-498d-a939-a1c3155e243e None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:04.948115 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799dff3f-3a7a-498d-a939-a1c3155e243e None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:04.948390 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799dff3f-3a7a-498d-a939-a1c3155e243e None admin] PATH_INFO: `/v3/services` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:04.949137 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799dff3f-3a7a-498d-a939-a1c3155e243e None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:27:04.949446 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799dff3f-3a7a-498d-a939-a1c3155e243e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:04.950794 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799dff3f-3a7a-498d-a939-a1c3155e243e None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:04.956214 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 466/927] 10.222.0.81 () {56 vars in 1114 bytes} [Wed Jul 20 22:27:04 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:27:04.957018 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799dff3f-3a7a-498d-a939-a1c3155e243e None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:04.964165 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 462/928] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:27:04 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 22:27:04.965034 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f388e2f-079a-447a-810b-31402e9f96ca None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:05.068577 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799dff3f-3a7a-498d-a939-a1c3155e243e None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:05.072122 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f388e2f-079a-447a-810b-31402e9f96ca None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:05.078345 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:05.104590 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fac94cf-9fb2-496f-a512-50eabb8f0994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:05.104590 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fac94cf-9fb2-496f-a512-50eabb8f0994 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:05.104590 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fac94cf-9fb2-496f-a512-50eabb8f0994 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:05.104893 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fac94cf-9fb2-496f-a512-50eabb8f0994 None admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:05.105548 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fac94cf-9fb2-496f-a512-50eabb8f0994 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:27:05.105857 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fac94cf-9fb2-496f-a512-50eabb8f0994 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:05.107244 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fac94cf-9fb2-496f-a512-50eabb8f0994 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:05.108194 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971b1876-c86d-46ae-bfa9-43b13231f451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:05.108910 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971b1876-c86d-46ae-bfa9-43b13231f451 None admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:05.109201 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971b1876-c86d-46ae-bfa9-43b13231f451 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:05.109490 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971b1876-c86d-46ae-bfa9-43b13231f451 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:05.110739 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971b1876-c86d-46ae-bfa9-43b13231f451 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:05.112138 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971b1876-c86d-46ae-bfa9-43b13231f451 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:05.116518 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:05.124044 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 467/929] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Jul 20 22:27:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:27:05.124044 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fac94cf-9fb2-496f-a512-50eabb8f0994 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:05.181325 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 463/930] 10.222.0.81 () {60 vars in 1144 bytes} [Wed Jul 20 22:27:04 2022] POST /identity/v3/endpoints => generated 346 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:27:05.181325 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971b1876-c86d-46ae-bfa9-43b13231f451 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:05.277473 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fac94cf-9fb2-496f-a512-50eabb8f0994 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:05.321385 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971b1876-c86d-46ae-bfa9-43b13231f451 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:05.326450 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74b26cc-027f-4d20-9276-1978562cfe59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:05.327315 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74b26cc-027f-4d20-9276-1978562cfe59 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:05.327598 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74b26cc-027f-4d20-9276-1978562cfe59 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:05.327873 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74b26cc-027f-4d20-9276-1978562cfe59 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:05.328535 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74b26cc-027f-4d20-9276-1978562cfe59 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:05.330139 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74b26cc-027f-4d20-9276-1978562cfe59 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:05.344565 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-e74b26cc-027f-4d20-9276-1978562cfe59 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 22:27:05.344565 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:27:05.344565 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 22:27:05.344565 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 22:27:05.344565 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 22:27:05.344565 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 22:27:05.344565 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 22:27:05.344565 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 22:27:05.344565 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 22:27:05.344565 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:27:05.344565 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 22:27:05.344565 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:27:05.344565 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:27:05.344565 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:27:05.344565 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:27:05.344565 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:27:05.344565 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:27:05.345587 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:27:05.345587 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:27:05.345587 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:27:05.345587 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:27:05.345587 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:27:05.345587 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:27:05.345587 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 22:27:05.345587 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 22:27:05.345587 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 22:27:05.345587 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 22:27:05.345587 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:27:05.345587 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:27:05.345587 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:27:05.345587 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:27:05.345587 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:27:05.345587 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:27:05.346419 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:27:05.346419 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:27:05.346419 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:27:05.346419 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:27:05.346419 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:27:05.346419 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:27:05.346419 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:27:05.346419 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:27:05.346419 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:27:05.346419 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:27:05.346419 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 22:27:05.346419 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 22:27:05.346419 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 22:27:05.346419 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:27:05.346419 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 468/931] 10.222.0.81 () {56 vars in 1126 bytes} [Wed Jul 20 22:27:05 2022] GET /identity/v3/services/object-store => generated 93 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:27:05.352653 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74b26cc-027f-4d20-9276-1978562cfe59 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:05.376752 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d30e09f-1832-44bb-a16b-433f2cad8a66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:05.377533 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d30e09f-1832-44bb-a16b-433f2cad8a66 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:05.377816 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d30e09f-1832-44bb-a16b-433f2cad8a66 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:05.378085 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d30e09f-1832-44bb-a16b-433f2cad8a66 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:05.385698 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d30e09f-1832-44bb-a16b-433f2cad8a66 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:05.387223 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d30e09f-1832-44bb-a16b-433f2cad8a66 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:05.387652 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:05.403051 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-9d30e09f-1832-44bb-a16b-433f2cad8a66 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 22:27:05.403051 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:27:05.403051 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:27:05.403051 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:27:05.403051 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:27:05.403051 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:27:05.403051 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:27:05.403051 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:27:05.403051 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:27:05.403051 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:27:05.403051 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:27:05.403051 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:27:05.403051 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 22:27:05.403051 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 22:27:05.403051 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 22:27:05.404008 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 22:27:05.404008 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:27:05.404008 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:27:05.404008 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:27:05.404008 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:27:05.404008 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:27:05.404008 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:27:05.404008 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:27:05.404008 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:27:05.404008 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:27:05.404008 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:27:05.404008 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:27:05.404008 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:27:05.404008 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:27:05.404008 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:27:05.404008 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:27:05.404008 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:27:05.404890 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 22:27:05.404890 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 22:27:05.404890 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 22:27:05.404890 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 22:27:05.404890 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 22:27:05.404890 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 22:27:05.404890 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 22:27:05.404890 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:27:05.405704 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 464/932] 10.222.0.81 () {56 vars in 1111 bytes} [Wed Jul 20 22:27:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:27:05.410764 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d30e09f-1832-44bb-a16b-433f2cad8a66 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:05.471403 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74b26cc-027f-4d20-9276-1978562cfe59 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:05.476572 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:05.507681 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35db5493-0464-4c2e-b4c2-eb5b6eeff17d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:05.508298 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35db5493-0464-4c2e-b4c2-eb5b6eeff17d None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:05.508638 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35db5493-0464-4c2e-b4c2-eb5b6eeff17d None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:05.508926 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35db5493-0464-4c2e-b4c2-eb5b6eeff17d None admin] PATH_INFO: `/v3/services` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:05.509721 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35db5493-0464-4c2e-b4c2-eb5b6eeff17d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:27:05.510055 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35db5493-0464-4c2e-b4c2-eb5b6eeff17d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:05.511775 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35db5493-0464-4c2e-b4c2-eb5b6eeff17d None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:05.517368 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 469/933] 10.222.0.81 () {56 vars in 1122 bytes} [Wed Jul 20 22:27:05 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 22:27:05.523138 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35db5493-0464-4c2e-b4c2-eb5b6eeff17d None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:05.554020 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d30e09f-1832-44bb-a16b-433f2cad8a66 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:05.600645 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bb9b3d-60e8-4f26-8732-a8f336cf4e3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:05.601627 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bb9b3d-60e8-4f26-8732-a8f336cf4e3e None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:05.602125 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bb9b3d-60e8-4f26-8732-a8f336cf4e3e None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:05.602587 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bb9b3d-60e8-4f26-8732-a8f336cf4e3e None admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:05.604120 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bb9b3d-60e8-4f26-8732-a8f336cf4e3e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:27:05.604443 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bb9b3d-60e8-4f26-8732-a8f336cf4e3e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:05.605901 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bb9b3d-60e8-4f26-8732-a8f336cf4e3e None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:05.619800 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 465/934] 10.222.0.81 () {56 vars in 1148 bytes} [Wed Jul 20 22:27:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 22:27:05.621096 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c26ec14-f6ea-4da8-8c17-59f96bd9102b None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:05.621464 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c26ec14-f6ea-4da8-8c17-59f96bd9102b None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:05.621753 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c26ec14-f6ea-4da8-8c17-59f96bd9102b None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:05.622976 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 466/935] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:27:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:27:05.625531 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:05.627338 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c26ec14-f6ea-4da8-8c17-59f96bd9102b None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:05.631813 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35db5493-0464-4c2e-b4c2-eb5b6eeff17d None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:05.668074 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd0bb33-0090-4b73-96d3-3311d5a44ae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:05.668800 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd0bb33-0090-4b73-96d3-3311d5a44ae3 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:05.669086 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd0bb33-0090-4b73-96d3-3311d5a44ae3 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:05.669420 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd0bb33-0090-4b73-96d3-3311d5a44ae3 None admin] PATH_INFO: `/v3/services` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:05.670190 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd0bb33-0090-4b73-96d3-3311d5a44ae3 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:27:05.670503 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd0bb33-0090-4b73-96d3-3311d5a44ae3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:05.671987 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd0bb33-0090-4b73-96d3-3311d5a44ae3 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:05.677796 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 470/936] 10.222.0.81 () {56 vars in 1122 bytes} [Wed Jul 20 22:27:05 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:27:05.678894 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0c3dc7-72d0-4bb0-83b9-6a7dde3df79b None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:05.679216 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0c3dc7-72d0-4bb0-83b9-6a7dde3df79b None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:05.679501 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0c3dc7-72d0-4bb0-83b9-6a7dde3df79b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:05.696952 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-cc0c3dc7-72d0-4bb0-83b9-6a7dde3df79b None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:27:05.698163 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-cc0c3dc7-72d0-4bb0-83b9-6a7dde3df79b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:27:05.738436 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c26ec14-f6ea-4da8-8c17-59f96bd9102b None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:05.745852 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:05.772407 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 471/937] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:27:05 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 22:27:05.773513 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0c3dc7-72d0-4bb0-83b9-6a7dde3df79b None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:05.782907 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac766dd4-2c4f-4e59-a93e-12fe474c0d15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:05.784466 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac766dd4-2c4f-4e59-a93e-12fe474c0d15 None admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:05.784848 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac766dd4-2c4f-4e59-a93e-12fe474c0d15 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:05.785201 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac766dd4-2c4f-4e59-a93e-12fe474c0d15 None admin] PATH_INFO: `/v3/projects/385edf2e59e546d090ace36600e29032/users/90915e1264324f16bc7bae7e46e0c97b/roles/a3e411d61a234343b961e8bf78a55a11` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:05.788426 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac766dd4-2c4f-4e59-a93e-12fe474c0d15 None admin] RBAC: Authorizing `identity:create_grant(project_id=385edf2e59e546d090ace36600e29032, user_id=90915e1264324f16bc7bae7e46e0c97b, role_id=a3e411d61a234343b961e8bf78a55a11)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:05.790806 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac766dd4-2c4f-4e59-a93e-12fe474c0d15 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:05.871018 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:05.881366 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0c3dc7-72d0-4bb0-83b9-6a7dde3df79b None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:05.886335 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 467/938] 10.222.0.81 () {58 vars in 1439 bytes} [Wed Jul 20 22:27:05 2022] PUT /identity/v3/projects/385edf2e59e546d090ace36600e29032/users/90915e1264324f16bc7bae7e46e0c97b/roles/a3e411d61a234343b961e8bf78a55a11 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 22:27:05.886959 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac766dd4-2c4f-4e59-a93e-12fe474c0d15 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:05.937593 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8e5c67-feab-471b-9a02-98949c27bbdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:05.940453 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8e5c67-feab-471b-9a02-98949c27bbdd None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:05.941050 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8e5c67-feab-471b-9a02-98949c27bbdd None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:05.942106 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8e5c67-feab-471b-9a02-98949c27bbdd None admin] PATH_INFO: `/v3/endpoints` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:05.943522 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8e5c67-feab-471b-9a02-98949c27bbdd None admin] RBAC: Adding query filter params (interface=public, service_id=29c9b2f63b364c7abd152381e6e9c44f, region_id=RegionOne) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:27:05.944031 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8e5c67-feab-471b-9a02-98949c27bbdd None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:05.946989 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8e5c67-feab-471b-9a02-98949c27bbdd None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:05.959365 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 472/939] 10.222.0.81 () {56 vars in 1251 bytes} [Wed Jul 20 22:27:05 2022] GET /identity/v3/endpoints?service_id=29c9b2f63b364c7abd152381e6e9c44f&interface=public®ion_id=RegionOne => generated 194 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:27:05.960548 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8e5c67-feab-471b-9a02-98949c27bbdd None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:06.023260 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac766dd4-2c4f-4e59-a93e-12fe474c0d15 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:06.066411 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce82bee-592b-4490-b53a-92f878d3663d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:06.068862 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce82bee-592b-4490-b53a-92f878d3663d None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:06.068862 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce82bee-592b-4490-b53a-92f878d3663d None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:06.068862 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce82bee-592b-4490-b53a-92f878d3663d None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:06.068862 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce82bee-592b-4490-b53a-92f878d3663d None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:06.069821 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce82bee-592b-4490-b53a-92f878d3663d None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:06.083846 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-9ce82bee-592b-4490-b53a-92f878d3663d None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 20 22:27:06.083846 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:27:06.083846 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 22:27:06.083846 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 22:27:06.083846 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 22:27:06.083846 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 22:27:06.083846 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 22:27:06.083846 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 22:27:06.083846 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 20 22:27:06.083846 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:27:06.083846 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 22:27:06.083846 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:27:06.083846 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:27:06.083846 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:27:06.083846 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:27:06.083846 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:27:06.084793 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:27:06.084793 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:27:06.084793 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:27:06.084793 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:27:06.084793 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:27:06.084793 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:27:06.084793 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:27:06.084793 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 22:27:06.084793 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 22:27:06.084793 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 22:27:06.084793 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 22:27:06.084793 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:27:06.084793 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:27:06.084793 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:27:06.084793 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:27:06.084793 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:27:06.085653 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:27:06.085653 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:27:06.085653 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:27:06.085653 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:27:06.085653 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:27:06.085653 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:27:06.085653 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:27:06.085653 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:27:06.085653 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:27:06.085653 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:27:06.085653 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:27:06.085653 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 22:27:06.085653 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 22:27:06.085653 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 20 22:27:06.085653 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:27:06.085653 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 468/940] 10.222.0.81 () {56 vars in 1132 bytes} [Wed Jul 20 22:27:05 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:27:06.090972 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce82bee-592b-4490-b53a-92f878d3663d None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:06.106651 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8e5c67-feab-471b-9a02-98949c27bbdd None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:06.132514 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:06.144850 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0aa6491-6142-4b08-8fd8-e8acb4434c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:06.145838 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aa6491-6142-4b08-8fd8-e8acb4434c41 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:06.146133 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aa6491-6142-4b08-8fd8-e8acb4434c41 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:06.146418 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aa6491-6142-4b08-8fd8-e8acb4434c41 None admin] PATH_INFO: `/v3/services` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:06.147185 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0aa6491-6142-4b08-8fd8-e8acb4434c41 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:06.148600 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0aa6491-6142-4b08-8fd8-e8acb4434c41 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:06.155456 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 473/941] 10.222.0.81 () {56 vars in 1087 bytes} [Wed Jul 20 22:27:05 2022] GET /identity/v3/services => generated 1497 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:27:06.221579 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce82bee-592b-4490-b53a-92f878d3663d None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:06.260658 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1321889-c96c-4321-a207-cf440e20b020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:06.261351 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1321889-c96c-4321-a207-cf440e20b020 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:06.261641 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1321889-c96c-4321-a207-cf440e20b020 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:06.261922 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1321889-c96c-4321-a207-cf440e20b020 None admin] PATH_INFO: `/v3/services` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:06.262648 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1321889-c96c-4321-a207-cf440e20b020 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:27:06.262963 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1321889-c96c-4321-a207-cf440e20b020 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:06.264382 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1321889-c96c-4321-a207-cf440e20b020 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:06.270949 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 469/942] 10.222.0.81 () {56 vars in 1126 bytes} [Wed Jul 20 22:27:06 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 22:27:06.276806 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0aa6491-6142-4b08-8fd8-e8acb4434c41 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:06.298834 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:06.387569 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0aa6491-6142-4b08-8fd8-e8acb4434c41 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:06.424114 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab91f34-5496-4795-bcb1-515310c44ee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:06.424812 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab91f34-5496-4795-bcb1-515310c44ee0 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:06.425098 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab91f34-5496-4795-bcb1-515310c44ee0 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:06.425425 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab91f34-5496-4795-bcb1-515310c44ee0 None admin] PATH_INFO: `/v3/services` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:06.426154 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab91f34-5496-4795-bcb1-515310c44ee0 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:27:06.426521 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab91f34-5496-4795-bcb1-515310c44ee0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:06.427977 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab91f34-5496-4795-bcb1-515310c44ee0 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:06.434090 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 474/943] 10.222.0.81 () {56 vars in 1126 bytes} [Wed Jul 20 22:27:06 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:27:06.439974 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1321889-c96c-4321-a207-cf440e20b020 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:06.539057 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:06.545809 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1321889-c96c-4321-a207-cf440e20b020 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:06.580481 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5207fa4-4dd6-48a0-8f26-e8a5006f6588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:06.581160 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5207fa4-4dd6-48a0-8f26-e8a5006f6588 None admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:06.581450 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5207fa4-4dd6-48a0-8f26-e8a5006f6588 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:06.581733 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5207fa4-4dd6-48a0-8f26-e8a5006f6588 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:06.582557 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5207fa4-4dd6-48a0-8f26-e8a5006f6588 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:06.584197 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5207fa4-4dd6-48a0-8f26-e8a5006f6588 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:06.688241 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 470/944] 10.222.0.81 () {60 vars in 1144 bytes} [Wed Jul 20 22:27:06 2022] POST /identity/v3/endpoints => generated 347 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:27:07.381828 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9415689c-36da-41c8-b74a-245dcc1b478d None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:07.382817 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9415689c-36da-41c8-b74a-245dcc1b478d None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:07.383339 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9415689c-36da-41c8-b74a-245dcc1b478d None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:07.385192 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 475/945] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:27:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:27:07.390616 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e4f8ca-76cd-4de4-8835-ef87a0dd415d None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:07.392030 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e4f8ca-76cd-4de4-8835-ef87a0dd415d None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:07.392530 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e4f8ca-76cd-4de4-8835-ef87a0dd415d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:07.410005 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-45e4f8ca-76cd-4de4-8835-ef87a0dd415d None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:27:07.411851 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-45e4f8ca-76cd-4de4-8835-ef87a0dd415d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:27:07.487468 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 471/946] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:27:07 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 22:27:07.510507 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9415689c-36da-41c8-b74a-245dcc1b478d None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:07.599032 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:07.618419 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9415689c-36da-41c8-b74a-245dcc1b478d None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:07.657567 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72abdfcd-e076-4878-830c-7595ffcb0c93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:07.658239 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72abdfcd-e076-4878-830c-7595ffcb0c93 None admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:07.658530 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72abdfcd-e076-4878-830c-7595ffcb0c93 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:07.658812 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72abdfcd-e076-4878-830c-7595ffcb0c93 None admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:07.659550 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72abdfcd-e076-4878-830c-7595ffcb0c93 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:07.661259 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72abdfcd-e076-4878-830c-7595ffcb0c93 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:07.662632 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: WARNING keystone.server.flask.common [None req-72abdfcd-e076-4878-830c-7595ffcb0c93 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 20 22:27:07.709318 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.sql.core [None req-72abdfcd-e076-4878-830c-7595ffcb0c93 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 20 22:27:07.709318 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 20 22:27:07.709318 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [parameters: {'id': '7b5e4b1d0ed7433da7e79c7a817ac230', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 20 22:27:07.709318 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80441) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 20 22:27:07.717378 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-72abdfcd-e076-4878-830c-7595ffcb0c93 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 7b5e4b1d0ed7433da7e79c7a817ac230.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 7b5e4b1d0ed7433da7e79c7a817ac230. Jul 20 22:27:07.717378 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:27:07.717378 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 20 22:27:07.717378 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 20 22:27:07.717378 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 20 22:27:07.717378 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 20 22:27:07.717378 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 20 22:27:07.717378 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application result = self._query(query) Jul 20 22:27:07.717378 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 20 22:27:07.717378 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application conn.query(q) Jul 20 22:27:07.717378 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 20 22:27:07.717378 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 20 22:27:07.717378 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 20 22:27:07.718172 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application result.read() Jul 20 22:27:07.718172 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 20 22:27:07.718172 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 20 22:27:07.718172 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 20 22:27:07.718172 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 20 22:27:07.718172 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 20 22:27:07.718172 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 20 22:27:07.718172 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 20 22:27:07.718172 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 20 22:27:07.718172 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 20 22:27:07.718172 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:27:07.718172 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 20 22:27:07.718172 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:27:07.718172 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:27:07.718172 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 20 22:27:07.718172 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 20 22:27:07.718172 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 20 22:27:07.718172 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 20 22:27:07.719256 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 20 22:27:07.719256 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application next(self.gen) Jul 20 22:27:07.719256 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 20 22:27:07.719256 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application yield resource Jul 20 22:27:07.719256 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 20 22:27:07.719256 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application next(self.gen) Jul 20 22:27:07.719256 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 20 22:27:07.719256 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application self.session.rollback() Jul 20 22:27:07.719256 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 20 22:27:07.719256 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application self.force_reraise() Jul 20 22:27:07.719256 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 20 22:27:07.719256 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise self.value Jul 20 22:27:07.719256 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 20 22:27:07.719256 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 20 22:27:07.719256 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 20 22:27:07.719256 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application session.commit() Jul 20 22:27:07.719256 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 20 22:27:07.720174 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 20 22:27:07.720174 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 20 22:27:07.720174 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application self._prepare_impl() Jul 20 22:27:07.720174 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 20 22:27:07.720174 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application self.session.flush() Jul 20 22:27:07.720174 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 20 22:27:07.720174 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application self._flush(objects) Jul 20 22:27:07.720174 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 20 22:27:07.720174 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 20 22:27:07.720174 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 20 22:27:07.720174 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application compat.raise_( Jul 20 22:27:07.720174 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 20 22:27:07.720174 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception Jul 20 22:27:07.720174 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 20 22:27:07.720174 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application flush_context.execute() Jul 20 22:27:07.720174 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 20 22:27:07.720174 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rec.execute(self) Jul 20 22:27:07.721085 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 20 22:27:07.721085 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 20 22:27:07.721085 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 20 22:27:07.721085 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 20 22:27:07.721085 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 20 22:27:07.721085 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 20 22:27:07.721085 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 20 22:27:07.721085 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 20 22:27:07.721085 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 20 22:27:07.721085 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 20 22:27:07.721085 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 20 22:27:07.721085 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 20 22:27:07.721085 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 20 22:27:07.721085 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 20 22:27:07.721085 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 20 22:27:07.721874 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 20 22:27:07.721874 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 20 22:27:07.721874 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception Jul 20 22:27:07.721874 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 20 22:27:07.721874 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 20 22:27:07.721874 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 20 22:27:07.721874 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 20 22:27:07.721874 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 20 22:27:07.721874 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application result = self._query(query) Jul 20 22:27:07.721874 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 20 22:27:07.721874 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application conn.query(q) Jul 20 22:27:07.721874 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 20 22:27:07.721874 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 20 22:27:07.721874 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 20 22:27:07.721874 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application result.read() Jul 20 22:27:07.721874 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 20 22:27:07.721874 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 20 22:27:07.722737 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 20 22:27:07.722737 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 20 22:27:07.722737 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 20 22:27:07.722737 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 20 22:27:07.722737 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 20 22:27:07.722737 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 20 22:27:07.722737 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 20 22:27:07.722737 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 20 22:27:07.722737 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [parameters: {'id': '7b5e4b1d0ed7433da7e79c7a817ac230', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 20 22:27:07.722737 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 20 22:27:07.722737 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:27:07.722737 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 22:27:07.722737 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:27:07.722737 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:27:07.722737 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 20 22:27:07.725230 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 20 22:27:07.725230 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 20 22:27:07.725230 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 20 22:27:07.725230 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 20 22:27:07.725230 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:27:07.725230 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 22:27:07.725230 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:27:07.725230 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:27:07.725230 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:27:07.725230 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:27:07.725230 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:27:07.725230 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:27:07.725230 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:27:07.725230 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:27:07.725230 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:27:07.725230 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:27:07.725230 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:27:07.727362 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:27:07.727362 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 20 22:27:07.727362 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 20 22:27:07.727362 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:27:07.727362 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:27:07.727362 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 20 22:27:07.727362 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 20 22:27:07.727362 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 7b5e4b1d0ed7433da7e79c7a817ac230. Jul 20 22:27:07.727362 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:27:07.727362 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 476/947] 10.222.0.81 () {60 vars in 1140 bytes} [Wed Jul 20 22:27:07 2022] POST /identity/v3/projects => generated 262 bytes in 209 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:27:07.735083 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e4f8ca-76cd-4de4-8835-ef87a0dd415d None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:07.815824 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:07.840983 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e4f8ca-76cd-4de4-8835-ef87a0dd415d None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:07.878147 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04060acb-0f83-488e-8292-1e923f75fc75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:07.878973 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04060acb-0f83-488e-8292-1e923f75fc75 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:07.879304 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04060acb-0f83-488e-8292-1e923f75fc75 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:07.879588 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04060acb-0f83-488e-8292-1e923f75fc75 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:07.886811 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04060acb-0f83-488e-8292-1e923f75fc75 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:07.889049 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04060acb-0f83-488e-8292-1e923f75fc75 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:07.904247 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-04060acb-0f83-488e-8292-1e923f75fc75 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 22:27:07.904247 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:27:07.904247 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:27:07.904247 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:27:07.904247 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:27:07.904247 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:27:07.904247 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:27:07.904247 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:27:07.904247 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:27:07.904247 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:27:07.904247 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:27:07.904247 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:27:07.904247 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 22:27:07.904247 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 22:27:07.904247 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 22:27:07.905147 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 22:27:07.905147 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:27:07.905147 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:27:07.905147 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:27:07.905147 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:27:07.905147 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:27:07.905147 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:27:07.905147 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:27:07.905147 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:27:07.905147 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:27:07.905147 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:27:07.905147 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:27:07.905147 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:27:07.905147 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:27:07.905147 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:27:07.905147 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:27:07.905147 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:27:07.906020 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 22:27:07.906020 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 22:27:07.906020 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 22:27:07.906020 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 22:27:07.906020 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 22:27:07.906020 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 22:27:07.906020 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 22:27:07.906020 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:27:07.906020 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 472/948] 10.222.0.81 () {56 vars in 1111 bytes} [Wed Jul 20 22:27:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:27:07.913405 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72abdfcd-e076-4878-830c-7595ffcb0c93 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:07.989322 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042553c0-ea7d-4975-84b9-f04f848adc13 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:07.989632 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042553c0-ea7d-4975-84b9-f04f848adc13 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:07.989892 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042553c0-ea7d-4975-84b9-f04f848adc13 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:07.991548 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 473/949] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:27:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 22:27:07.996419 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbd10ea-dc53-40f5-bdd1-250d6dcd6336 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:07.996719 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbd10ea-dc53-40f5-bdd1-250d6dcd6336 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:07.996981 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbd10ea-dc53-40f5-bdd1-250d6dcd6336 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:08.018649 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-cfbd10ea-dc53-40f5-bdd1-250d6dcd6336 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:27:08.020964 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-cfbd10ea-dc53-40f5-bdd1-250d6dcd6336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:27:08.025811 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72abdfcd-e076-4878-830c-7595ffcb0c93 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:08.065146 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d820c7d8-3171-4492-be09-efcf83eed6f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:08.066092 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d820c7d8-3171-4492-be09-efcf83eed6f6 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:08.066367 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d820c7d8-3171-4492-be09-efcf83eed6f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:08.066787 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d820c7d8-3171-4492-be09-efcf83eed6f6 None admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:08.067889 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d820c7d8-3171-4492-be09-efcf83eed6f6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:27:08.068298 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d820c7d8-3171-4492-be09-efcf83eed6f6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:08.070931 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d820c7d8-3171-4492-be09-efcf83eed6f6 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:08.083108 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:08.095872 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 477/950] 10.222.0.81 () {56 vars in 1112 bytes} [Wed Jul 20 22:27:07 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:27:08.117877 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 474/951] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:27:07 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 22:27:08.136951 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d820c7d8-3171-4492-be09-efcf83eed6f6 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:08.267333 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d820c7d8-3171-4492-be09-efcf83eed6f6 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:08.305081 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5595249-0aa9-4649-a1de-01563145ba14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:08.305920 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5595249-0aa9-4649-a1de-01563145ba14 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:08.306205 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5595249-0aa9-4649-a1de-01563145ba14 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:08.306486 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5595249-0aa9-4649-a1de-01563145ba14 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:08.317316 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5595249-0aa9-4649-a1de-01563145ba14 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 22:27:08.325906 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-b5595249-0aa9-4649-a1de-01563145ba14 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 22:27:08.325906 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:27:08.325906 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:27:08.325906 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:27:08.325906 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:27:08.325906 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:27:08.325906 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:27:08.325906 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:27:08.325906 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:27:08.325906 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:27:08.325906 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:27:08.325906 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:27:08.325906 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 22:27:08.325906 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 22:27:08.325906 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 22:27:08.325906 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 22:27:08.326945 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 22:27:08.326945 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 22:27:08.326945 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 22:27:08.326945 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 22:27:08.326945 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:27:08.326945 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:27:08.326945 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:27:08.326945 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:27:08.326945 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:27:08.326945 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:27:08.326945 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:27:08.326945 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:27:08.326945 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:27:08.326945 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:27:08.326945 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:27:08.326945 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:27:08.328008 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:27:08.328008 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:27:08.328008 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:27:08.328008 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:27:08.328008 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 22:27:08.328008 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 22:27:08.328008 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 22:27:08.328008 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 22:27:08.328008 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 22:27:08.328008 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 22:27:08.328008 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 22:27:08.328008 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:27:08.328008 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 478/952] 10.222.0.81 () {56 vars in 1120 bytes} [Wed Jul 20 22:27:08 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:27:08.335946 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbd10ea-dc53-40f5-bdd1-250d6dcd6336 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:08.365936 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa894e0a-d419-4688-873c-68f0d0e6b868 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:08.366126 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa894e0a-d419-4688-873c-68f0d0e6b868 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:08.366403 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa894e0a-d419-4688-873c-68f0d0e6b868 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:08.368244 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 479/953] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:27:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:27:08.373483 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc8baae-cbe4-437c-bf58-0bde4c3128e8 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:08.373776 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc8baae-cbe4-437c-bf58-0bde4c3128e8 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:08.374062 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc8baae-cbe4-437c-bf58-0bde4c3128e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:08.403809 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-bcc8baae-cbe4-437c-bf58-0bde4c3128e8 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:27:08.405349 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-bcc8baae-cbe4-437c-bf58-0bde4c3128e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:27:08.427404 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:08.452254 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbd10ea-dc53-40f5-bdd1-250d6dcd6336 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:08.473769 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:08.488516 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 480/954] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:27:08 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 22:27:08.493364 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece705d6-67d9-422d-b8e5-57fb868f4d50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:08.494056 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece705d6-67d9-422d-b8e5-57fb868f4d50 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:08.494345 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece705d6-67d9-422d-b8e5-57fb868f4d50 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:08.494624 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece705d6-67d9-422d-b8e5-57fb868f4d50 None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:08.495414 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece705d6-67d9-422d-b8e5-57fb868f4d50 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:27:08.495750 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece705d6-67d9-422d-b8e5-57fb868f4d50 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:08.497388 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece705d6-67d9-422d-b8e5-57fb868f4d50 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:08.506489 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 475/955] 10.222.0.81 () {56 vars in 1115 bytes} [Wed Jul 20 22:27:08 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:27:08.512096 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc8baae-cbe4-437c-bf58-0bde4c3128e8 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:08.512500 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece705d6-67d9-422d-b8e5-57fb868f4d50 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:08.621437 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece705d6-67d9-422d-b8e5-57fb868f4d50 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:08.645543 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc8baae-cbe4-437c-bf58-0bde4c3128e8 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:08.650120 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:08.662481 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deafb0f-2263-426d-b47d-999f728480f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:08.663531 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deafb0f-2263-426d-b47d-999f728480f9 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:08.663960 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deafb0f-2263-426d-b47d-999f728480f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:08.664512 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deafb0f-2263-426d-b47d-999f728480f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:08.675820 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6deafb0f-2263-426d-b47d-999f728480f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:08.677446 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6deafb0f-2263-426d-b47d-999f728480f9 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:08.687485 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9664158-634c-41d6-8728-c1c11c596164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:08.688846 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9664158-634c-41d6-8728-c1c11c596164 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:08.689134 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9664158-634c-41d6-8728-c1c11c596164 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:08.689416 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9664158-634c-41d6-8728-c1c11c596164 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:08.690070 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9664158-634c-41d6-8728-c1c11c596164 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:08.691509 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9664158-634c-41d6-8728-c1c11c596164 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:08.698698 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-6deafb0f-2263-426d-b47d-999f728480f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:27:08.698698 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:27:08.698698 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:27:08.698698 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:27:08.698698 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:27:08.698698 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:27:08.698698 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:27:08.698698 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:27:08.698698 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:27:08.698698 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:27:08.698698 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:27:08.698698 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:27:08.698698 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 22:27:08.698698 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 22:27:08.698698 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 22:27:08.699791 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 22:27:08.699791 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:27:08.699791 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:27:08.699791 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:27:08.699791 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:27:08.699791 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:27:08.699791 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:27:08.699791 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:27:08.699791 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:27:08.699791 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:27:08.699791 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:27:08.699791 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:27:08.699791 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:27:08.699791 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:27:08.699791 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:27:08.699791 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:27:08.699791 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:27:08.700871 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 22:27:08.700871 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 22:27:08.700871 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:27:08.700871 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:27:08.707341 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-d9664158-634c-41d6-8728-c1c11c596164 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 22:27:08.707341 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:27:08.707341 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 22:27:08.707341 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 22:27:08.707341 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 22:27:08.707341 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 22:27:08.707341 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 22:27:08.707341 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 22:27:08.707341 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 22:27:08.707341 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:27:08.707341 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 22:27:08.707341 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:27:08.707341 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:27:08.707341 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:27:08.707341 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:27:08.707341 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:27:08.707341 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:27:08.708635 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:27:08.708635 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:27:08.708635 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:27:08.708635 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:27:08.708635 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:27:08.708635 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:27:08.708635 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 22:27:08.708635 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 22:27:08.708635 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 22:27:08.708635 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 22:27:08.708635 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:27:08.708635 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:27:08.708635 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:27:08.708635 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:27:08.708635 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:27:08.708635 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:27:08.709507 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:27:08.709507 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:27:08.709507 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:27:08.709507 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:27:08.709507 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:27:08.709507 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:27:08.709507 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:27:08.709507 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:27:08.709507 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:27:08.709507 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:27:08.709507 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 22:27:08.709507 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 22:27:08.709507 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 22:27:08.709507 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:27:08.710358 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 476/957] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Jul 20 22:27:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:27:08.711561 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 481/957] 10.222.0.81 () {56 vars in 1126 bytes} [Wed Jul 20 22:27:08 2022] GET /identity/v3/services/object-store => generated 93 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:27:08.712436 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9664158-634c-41d6-8728-c1c11c596164 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:08.712732 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deafb0f-2263-426d-b47d-999f728480f9 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:08.784654 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:08.839809 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9664158-634c-41d6-8728-c1c11c596164 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:08.843173 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deafb0f-2263-426d-b47d-999f728480f9 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:08.879545 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe1c678-0704-41b5-8824-ee6163f1301a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:08.879962 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaf976d-5346-49dd-802e-5188fe768c1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:08.880283 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe1c678-0704-41b5-8824-ee6163f1301a None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:08.880634 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaf976d-5346-49dd-802e-5188fe768c1c None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:08.880733 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe1c678-0704-41b5-8824-ee6163f1301a None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:08.880920 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaf976d-5346-49dd-802e-5188fe768c1c None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:08.881016 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe1c678-0704-41b5-8824-ee6163f1301a None admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:08.881195 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaf976d-5346-49dd-802e-5188fe768c1c None admin] PATH_INFO: `/v3/services` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:08.881838 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe1c678-0704-41b5-8824-ee6163f1301a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:27:08.881932 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebaf976d-5346-49dd-802e-5188fe768c1c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:27:08.882190 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebaf976d-5346-49dd-802e-5188fe768c1c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:08.882295 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe1c678-0704-41b5-8824-ee6163f1301a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:08.883578 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebaf976d-5346-49dd-802e-5188fe768c1c None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:08.883676 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe1c678-0704-41b5-8824-ee6163f1301a None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:08.891523 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 477/958] 10.222.0.81 () {56 vars in 1122 bytes} [Wed Jul 20 22:27:08 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:27:08.891523 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb1557c-536d-4a02-9bac-f036847bf9cf None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:08.891523 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb1557c-536d-4a02-9bac-f036847bf9cf None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:08.891523 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb1557c-536d-4a02-9bac-f036847bf9cf None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:08.892058 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 478/959] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:27:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:27:08.897224 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be9f103-bdc8-4d8b-9fbf-6e49d2ef00b6 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:08.897686 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be9f103-bdc8-4d8b-9fbf-6e49d2ef00b6 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:08.898117 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be9f103-bdc8-4d8b-9fbf-6e49d2ef00b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:08.898698 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 482/960] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Jul 20 22:27:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:27:08.899567 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe1c678-0704-41b5-8824-ee6163f1301a None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:08.918859 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-5be9f103-bdc8-4d8b-9fbf-6e49d2ef00b6 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:27:08.920109 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-5be9f103-bdc8-4d8b-9fbf-6e49d2ef00b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:27:08.962126 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:08.989917 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 479/961] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:27:08 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 22:27:08.991146 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be9f103-bdc8-4d8b-9fbf-6e49d2ef00b6 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:09.025881 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe1c678-0704-41b5-8824-ee6163f1301a None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:09.042398 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:09.068922 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ef1b62-8bd9-4a65-adfd-fe0c5595550c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:09.069975 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ef1b62-8bd9-4a65-adfd-fe0c5595550c None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:09.070493 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ef1b62-8bd9-4a65-adfd-fe0c5595550c None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:09.071147 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ef1b62-8bd9-4a65-adfd-fe0c5595550c None admin] PATH_INFO: `/v3/services` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:09.071935 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ef1b62-8bd9-4a65-adfd-fe0c5595550c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:27:09.072253 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ef1b62-8bd9-4a65-adfd-fe0c5595550c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:09.073894 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ef1b62-8bd9-4a65-adfd-fe0c5595550c None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:09.084841 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 483/962] 10.222.0.81 () {56 vars in 1122 bytes} [Wed Jul 20 22:27:08 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:27:09.084841 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ef1b62-8bd9-4a65-adfd-fe0c5595550c None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:09.100439 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be9f103-bdc8-4d8b-9fbf-6e49d2ef00b6 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:09.135239 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c345e90b-21bd-4971-92ae-e0c2ad896d92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:09.135999 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c345e90b-21bd-4971-92ae-e0c2ad896d92 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:09.136286 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c345e90b-21bd-4971-92ae-e0c2ad896d92 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:09.136567 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c345e90b-21bd-4971-92ae-e0c2ad896d92 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:09.143645 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c345e90b-21bd-4971-92ae-e0c2ad896d92 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:09.145081 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c345e90b-21bd-4971-92ae-e0c2ad896d92 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:09.161753 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-c345e90b-21bd-4971-92ae-e0c2ad896d92 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 22:27:09.161753 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:27:09.161753 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:27:09.161753 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:27:09.161753 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:27:09.161753 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:27:09.161753 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:27:09.161753 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:27:09.161753 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:27:09.161753 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:27:09.161753 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:27:09.161753 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:27:09.161753 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 22:27:09.161753 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 22:27:09.161753 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 22:27:09.161753 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 22:27:09.162694 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:27:09.162694 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:27:09.162694 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 22:27:09.162694 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 22:27:09.162694 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 22:27:09.162694 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 22:27:09.162694 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:27:09.162694 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:27:09.162694 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:27:09.162694 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:27:09.162694 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:27:09.162694 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:27:09.162694 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:27:09.162694 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:27:09.162694 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:27:09.162694 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:27:09.162694 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:27:09.162694 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:27:09.168579 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:27:09.168579 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:27:09.168579 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 22:27:09.168579 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 22:27:09.168579 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 22:27:09.168579 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 22:27:09.168579 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 22:27:09.168579 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 22:27:09.168579 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 22:27:09.168579 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:27:09.168579 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 480/963] 10.222.0.81 () {56 vars in 1117 bytes} [Wed Jul 20 22:27:08 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:27:09.168579 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c345e90b-21bd-4971-92ae-e0c2ad896d92 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:09.207160 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ef1b62-8bd9-4a65-adfd-fe0c5595550c None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:09.214747 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:09.246139 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f82fa5-2f69-419f-8678-e5666317c31c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:09.246953 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f82fa5-2f69-419f-8678-e5666317c31c None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:09.247274 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f82fa5-2f69-419f-8678-e5666317c31c None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:09.247555 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f82fa5-2f69-419f-8678-e5666317c31c None admin] PATH_INFO: `/v3/services/compute` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:09.248228 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f82fa5-2f69-419f-8678-e5666317c31c None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:09.249581 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f82fa5-2f69-419f-8678-e5666317c31c None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:09.262040 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-a1f82fa5-2f69-419f-8678-e5666317c31c None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 20 22:27:09.262040 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:27:09.262040 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 22:27:09.262040 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 22:27:09.262040 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 22:27:09.262040 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 22:27:09.262040 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 22:27:09.262040 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 22:27:09.262040 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 20 22:27:09.262040 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:27:09.262040 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 22:27:09.262040 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:27:09.262040 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:27:09.262040 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:27:09.262040 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:27:09.262040 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:27:09.262040 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:27:09.263075 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:27:09.263075 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:27:09.263075 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:27:09.263075 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:27:09.263075 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:27:09.263075 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:27:09.263075 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 22:27:09.263075 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 22:27:09.263075 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 22:27:09.263075 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 22:27:09.263075 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:27:09.263075 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:27:09.263075 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:27:09.263075 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:27:09.263075 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:27:09.263075 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:27:09.263937 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:27:09.263937 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:27:09.263937 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:27:09.263937 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:27:09.263937 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:27:09.263937 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:27:09.263937 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:27:09.263937 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:27:09.263937 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:27:09.263937 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:27:09.263937 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 22:27:09.263937 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 22:27:09.263937 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 20 22:27:09.263937 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:27:09.263937 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 484/964] 10.222.0.81 () {56 vars in 1111 bytes} [Wed Jul 20 22:27:09 2022] GET /identity/v3/services/compute => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:27:09.264772 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f82fa5-2f69-419f-8678-e5666317c31c None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:09.287184 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c345e90b-21bd-4971-92ae-e0c2ad896d92 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:09.309986 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:09.323932 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c221d8-a9bc-4f3c-856c-3bc851eb052e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:09.324900 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c221d8-a9bc-4f3c-856c-3bc851eb052e None admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:09.325349 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c221d8-a9bc-4f3c-856c-3bc851eb052e None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:09.325629 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c221d8-a9bc-4f3c-856c-3bc851eb052e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:09.326366 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c221d8-a9bc-4f3c-856c-3bc851eb052e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:09.327758 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c221d8-a9bc-4f3c-856c-3bc851eb052e None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:09.369730 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f82fa5-2f69-419f-8678-e5666317c31c None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:09.401394 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 481/965] 10.222.0.81 () {60 vars in 1144 bytes} [Wed Jul 20 22:27:09 2022] POST /identity/v3/endpoints => generated 349 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:27:09.402523 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c221d8-a9bc-4f3c-856c-3bc851eb052e None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:09.432739 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02983611-8f2b-4732-b7a4-187fb8cc1d78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:09.433614 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02983611-8f2b-4732-b7a4-187fb8cc1d78 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:09.433804 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02983611-8f2b-4732-b7a4-187fb8cc1d78 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:09.434234 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02983611-8f2b-4732-b7a4-187fb8cc1d78 None admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:09.435194 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02983611-8f2b-4732-b7a4-187fb8cc1d78 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:27:09.435517 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02983611-8f2b-4732-b7a4-187fb8cc1d78 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:09.437596 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02983611-8f2b-4732-b7a4-187fb8cc1d78 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:09.466542 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 485/966] 10.222.0.81 () {56 vars in 1149 bytes} [Wed Jul 20 22:27:09 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:27:09.480294 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02983611-8f2b-4732-b7a4-187fb8cc1d78 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:09.532596 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:09.544066 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c221d8-a9bc-4f3c-856c-3bc851eb052e None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:09.582454 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3091516-0870-46bb-9619-6c09a5114ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:09.583184 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3091516-0870-46bb-9619-6c09a5114ef4 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:09.583521 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3091516-0870-46bb-9619-6c09a5114ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:09.583804 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3091516-0870-46bb-9619-6c09a5114ef4 None admin] PATH_INFO: `/v3/services` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:09.584621 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3091516-0870-46bb-9619-6c09a5114ef4 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:27:09.584936 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3091516-0870-46bb-9619-6c09a5114ef4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:09.586322 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3091516-0870-46bb-9619-6c09a5114ef4 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:09.591845 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 482/967] 10.222.0.81 () {56 vars in 1112 bytes} [Wed Jul 20 22:27:09 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:27:09.598908 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3091516-0870-46bb-9619-6c09a5114ef4 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:09.613672 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:09.623635 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02983611-8f2b-4732-b7a4-187fb8cc1d78 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:09.665128 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdb2768-c4ab-408b-9ee0-a33f6fcde907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:09.666041 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdb2768-c4ab-408b-9ee0-a33f6fcde907 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:09.666490 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdb2768-c4ab-408b-9ee0-a33f6fcde907 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:09.666960 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdb2768-c4ab-408b-9ee0-a33f6fcde907 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:09.729821 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3091516-0870-46bb-9619-6c09a5114ef4 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:09.737170 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdb2768-c4ab-408b-9ee0-a33f6fcde907 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:09.738910 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdb2768-c4ab-408b-9ee0-a33f6fcde907 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:09.775801 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71febc2-8bf1-4a1f-9758-4078a85f9881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:09.776484 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71febc2-8bf1-4a1f-9758-4078a85f9881 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:09.776772 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71febc2-8bf1-4a1f-9758-4078a85f9881 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:09.777053 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71febc2-8bf1-4a1f-9758-4078a85f9881 None admin] PATH_INFO: `/v3/services` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:09.778025 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71febc2-8bf1-4a1f-9758-4078a85f9881 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:27:09.778345 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71febc2-8bf1-4a1f-9758-4078a85f9881 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:09.780663 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71febc2-8bf1-4a1f-9758-4078a85f9881 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:09.787406 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 483/968] 10.222.0.81 () {56 vars in 1112 bytes} [Wed Jul 20 22:27:09 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:27:09.857801 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:09.868117 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 486/969] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Jul 20 22:27:09 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 20 22:27:09.873889 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71febc2-8bf1-4a1f-9758-4078a85f9881 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:09.898704 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:09.987084 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71febc2-8bf1-4a1f-9758-4078a85f9881 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:10.021322 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44db90c-954f-4e93-b60c-906d785aa933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:10.022140 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44db90c-954f-4e93-b60c-906d785aa933 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:10.022425 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44db90c-954f-4e93-b60c-906d785aa933 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:10.022703 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44db90c-954f-4e93-b60c-906d785aa933 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:10.034477 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44db90c-954f-4e93-b60c-906d785aa933 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:10.036226 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44db90c-954f-4e93-b60c-906d785aa933 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:10.055339 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-b44db90c-954f-4e93-b60c-906d785aa933 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:27:10.055339 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:27:10.055339 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:27:10.055339 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:27:10.055339 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:27:10.055339 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:27:10.055339 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:27:10.055339 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:27:10.055339 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:27:10.055339 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:27:10.055339 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:27:10.055339 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:27:10.055339 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 22:27:10.055339 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 22:27:10.055339 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 22:27:10.056266 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 22:27:10.056266 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:27:10.056266 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:27:10.056266 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:27:10.056266 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:27:10.056266 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:27:10.056266 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:27:10.056266 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:27:10.056266 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:27:10.056266 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:27:10.056266 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:27:10.056266 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:27:10.056266 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:27:10.056266 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:27:10.056266 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:27:10.056266 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:27:10.056266 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:27:10.057159 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 22:27:10.057159 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 22:27:10.057159 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:27:10.057159 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:27:10.057159 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 484/970] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Jul 20 22:27:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:27:10.062362 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdb2768-c4ab-408b-9ee0-a33f6fcde907 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:10.173084 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdb2768-c4ab-408b-9ee0-a33f6fcde907 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:10.213117 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315e7981-1909-46d8-a7b9-b36d7d82f247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:10.213926 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315e7981-1909-46d8-a7b9-b36d7d82f247 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:10.214355 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315e7981-1909-46d8-a7b9-b36d7d82f247 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:10.214764 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315e7981-1909-46d8-a7b9-b36d7d82f247 None admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:10.215718 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315e7981-1909-46d8-a7b9-b36d7d82f247 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:27:10.216045 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315e7981-1909-46d8-a7b9-b36d7d82f247 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:10.217432 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315e7981-1909-46d8-a7b9-b36d7d82f247 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:10.232022 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 487/971] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Jul 20 22:27:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:27:10.240364 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44db90c-954f-4e93-b60c-906d785aa933 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:10.314894 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:10.346009 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44db90c-954f-4e93-b60c-906d785aa933 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:10.390093 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2bb2be-59c4-4e27-bd60-a64a60b34c1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:10.390908 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2bb2be-59c4-4e27-bd60-a64a60b34c1e None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:10.391282 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2bb2be-59c4-4e27-bd60-a64a60b34c1e None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:10.391565 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2bb2be-59c4-4e27-bd60-a64a60b34c1e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:10.398323 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2bb2be-59c4-4e27-bd60-a64a60b34c1e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:10.399784 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2bb2be-59c4-4e27-bd60-a64a60b34c1e None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:10.414006 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d18118-eaf6-4459-ba02-60737228a6f4 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:10.414301 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d18118-eaf6-4459-ba02-60737228a6f4 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:10.414565 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-8a2bb2be-59c4-4e27-bd60-a64a60b34c1e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 22:27:10.414565 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:27:10.414565 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:27:10.414565 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:27:10.414565 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:27:10.414565 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:27:10.414565 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:27:10.414565 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:27:10.414565 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:27:10.414565 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:27:10.414565 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:27:10.414565 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:27:10.414565 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 22:27:10.414565 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 22:27:10.414565 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 22:27:10.415441 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 22:27:10.415441 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:27:10.415441 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:27:10.415441 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:27:10.415441 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:27:10.415441 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:27:10.415441 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:27:10.415441 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:27:10.415441 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:27:10.415441 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:27:10.415441 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:27:10.415441 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:27:10.415441 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:27:10.415441 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:27:10.415441 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:27:10.415441 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:27:10.415441 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:27:10.416331 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 22:27:10.416331 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 22:27:10.416331 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 22:27:10.416331 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 22:27:10.416331 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 22:27:10.416331 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 22:27:10.416331 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 22:27:10.416331 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:27:10.416749 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d18118-eaf6-4459-ba02-60737228a6f4 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:10.416827 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 485/972] 10.222.0.81 () {56 vars in 1111 bytes} [Wed Jul 20 22:27:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:27:10.417338 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 488/973] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:27:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:27:10.421369 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d18118-eaf6-4459-ba02-60737228a6f4 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:10.421655 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7e0e83-6765-487f-ad0e-7f064786c693 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:10.421948 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7e0e83-6765-487f-ad0e-7f064786c693 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:10.422750 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7e0e83-6765-487f-ad0e-7f064786c693 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:10.441246 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-5e7e0e83-6765-487f-ad0e-7f064786c693 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:27:10.442455 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-5e7e0e83-6765-487f-ad0e-7f064786c693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:27:10.503482 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:10.526623 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 486/974] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:27:10 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 22:27:10.551910 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7e0e83-6765-487f-ad0e-7f064786c693 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:10.555907 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d18118-eaf6-4459-ba02-60737228a6f4 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:10.602265 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86988cd-80c5-48c6-bd6f-66e3bf71489b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:10.603586 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86988cd-80c5-48c6-bd6f-66e3bf71489b None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:10.604443 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86988cd-80c5-48c6-bd6f-66e3bf71489b None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:10.604954 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86988cd-80c5-48c6-bd6f-66e3bf71489b None admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:10.605957 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86988cd-80c5-48c6-bd6f-66e3bf71489b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:27:10.606474 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86988cd-80c5-48c6-bd6f-66e3bf71489b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:10.608973 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86988cd-80c5-48c6-bd6f-66e3bf71489b None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:10.614866 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:10.627039 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 489/975] 10.222.0.81 () {56 vars in 1148 bytes} [Wed Jul 20 22:27:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 22:27:10.639032 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86988cd-80c5-48c6-bd6f-66e3bf71489b None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:10.659825 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7e0e83-6765-487f-ad0e-7f064786c693 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:10.701274 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3ccc64-0db1-4772-8be8-8dc45dd68366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:10.701967 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3ccc64-0db1-4772-8be8-8dc45dd68366 None admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:10.702262 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3ccc64-0db1-4772-8be8-8dc45dd68366 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:10.702549 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3ccc64-0db1-4772-8be8-8dc45dd68366 None admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:10.703328 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3ccc64-0db1-4772-8be8-8dc45dd68366 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:10.704830 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3ccc64-0db1-4772-8be8-8dc45dd68366 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:10.706158 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: WARNING keystone.server.flask.common [None req-7b3ccc64-0db1-4772-8be8-8dc45dd68366 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 20 22:27:10.749525 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.sql.core [None req-7b3ccc64-0db1-4772-8be8-8dc45dd68366 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 20 22:27:10.749525 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 20 22:27:10.749525 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [parameters: {'user_id': '76d0879b55b440c687a697701c37b6f7', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 20 22:27:10.749525 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80440) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 20 22:27:10.756336 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86988cd-80c5-48c6-bd6f-66e3bf71489b None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:10.772822 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:10.775805 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-7b3ccc64-0db1-4772-8be8-8dc45dd68366 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 20 22:27:10.775805 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:27:10.775805 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 20 22:27:10.775805 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 20 22:27:10.775805 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 20 22:27:10.775805 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 20 22:27:10.775805 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 20 22:27:10.775805 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application result = self._query(query) Jul 20 22:27:10.775805 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 20 22:27:10.775805 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application conn.query(q) Jul 20 22:27:10.775805 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 20 22:27:10.775805 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 20 22:27:10.775805 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 20 22:27:10.775805 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application result.read() Jul 20 22:27:10.775805 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 20 22:27:10.776868 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 20 22:27:10.776868 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 20 22:27:10.776868 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 20 22:27:10.776868 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 20 22:27:10.776868 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 20 22:27:10.776868 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 20 22:27:10.776868 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 20 22:27:10.776868 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 20 22:27:10.776868 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:27:10.776868 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 20 22:27:10.776868 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:27:10.776868 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:27:10.776868 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 20 22:27:10.776868 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 20 22:27:10.776868 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 20 22:27:10.776868 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 20 22:27:10.776868 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 20 22:27:10.776868 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application next(self.gen) Jul 20 22:27:10.778069 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 20 22:27:10.778069 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application yield resource Jul 20 22:27:10.778069 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 20 22:27:10.778069 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application next(self.gen) Jul 20 22:27:10.778069 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 20 22:27:10.778069 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application self.session.rollback() Jul 20 22:27:10.778069 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 20 22:27:10.778069 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application self.force_reraise() Jul 20 22:27:10.778069 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 20 22:27:10.778069 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise self.value Jul 20 22:27:10.778069 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 20 22:27:10.778069 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 20 22:27:10.778069 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 20 22:27:10.778069 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application session.commit() Jul 20 22:27:10.778069 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 20 22:27:10.778069 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 20 22:27:10.778069 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 20 22:27:10.779615 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application self._prepare_impl() Jul 20 22:27:10.779615 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 20 22:27:10.779615 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application self.session.flush() Jul 20 22:27:10.779615 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 20 22:27:10.779615 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application self._flush(objects) Jul 20 22:27:10.779615 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 20 22:27:10.779615 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 20 22:27:10.779615 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 20 22:27:10.779615 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application compat.raise_( Jul 20 22:27:10.779615 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 20 22:27:10.779615 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception Jul 20 22:27:10.779615 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 20 22:27:10.779615 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application flush_context.execute() Jul 20 22:27:10.779615 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 20 22:27:10.779615 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rec.execute(self) Jul 20 22:27:10.779615 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 20 22:27:10.779615 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 20 22:27:10.782692 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 20 22:27:10.782692 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 20 22:27:10.782692 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 20 22:27:10.782692 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 20 22:27:10.782692 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 20 22:27:10.782692 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 20 22:27:10.782692 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 20 22:27:10.782692 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 20 22:27:10.782692 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 20 22:27:10.782692 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 20 22:27:10.782692 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 20 22:27:10.782692 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 20 22:27:10.782692 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 20 22:27:10.782692 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 20 22:27:10.782692 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 20 22:27:10.782692 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception Jul 20 22:27:10.784742 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 20 22:27:10.784742 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 20 22:27:10.784742 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 20 22:27:10.784742 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 20 22:27:10.784742 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 20 22:27:10.784742 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application result = self._query(query) Jul 20 22:27:10.784742 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 20 22:27:10.784742 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application conn.query(q) Jul 20 22:27:10.784742 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 20 22:27:10.784742 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 20 22:27:10.784742 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 20 22:27:10.784742 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application result.read() Jul 20 22:27:10.784742 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 20 22:27:10.784742 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 20 22:27:10.784742 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 20 22:27:10.784742 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 20 22:27:10.787766 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 20 22:27:10.787766 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 20 22:27:10.787766 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 20 22:27:10.787766 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 20 22:27:10.787766 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 20 22:27:10.787766 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 20 22:27:10.787766 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [parameters: {'user_id': '76d0879b55b440c687a697701c37b6f7', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 20 22:27:10.787766 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 20 22:27:10.787766 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:27:10.787766 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 22:27:10.787766 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:27:10.787766 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:27:10.787766 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:27:10.787766 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:27:10.787766 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:27:10.789747 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:27:10.789747 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:27:10.789747 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:27:10.789747 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:27:10.789747 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:27:10.789747 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:27:10.789747 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:27:10.789747 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 20 22:27:10.789747 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 20 22:27:10.789747 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:27:10.789747 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:27:10.789747 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 22:27:10.789747 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 22:27:10.789747 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 22:27:10.789747 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 22:27:10.789747 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 20 22:27:10.789747 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 20 22:27:10.791843 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 20 22:27:10.791843 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 20 22:27:10.791843 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 20 22:27:10.791843 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 20 22:27:10.791843 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 20 22:27:10.791843 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:27:10.791843 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 487/976] 10.222.0.81 () {60 vars in 1131 bytes} [Wed Jul 20 22:27:10 2022] POST /identity/v3/users => generated 160 bytes in 229 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:27:10.791843 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3ccc64-0db1-4772-8be8-8dc45dd68366 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:10.810335 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62f2dab-3ab7-46e6-b7d8-086b735bc137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:10.811640 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62f2dab-3ab7-46e6-b7d8-086b735bc137 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:10.812142 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62f2dab-3ab7-46e6-b7d8-086b735bc137 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:10.812607 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62f2dab-3ab7-46e6-b7d8-086b735bc137 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:10.813873 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62f2dab-3ab7-46e6-b7d8-086b735bc137 None admin] RBAC: Adding query filter params (role.id=a3e411d61a234343b961e8bf78a55a11, scope.project.id=385edf2e59e546d090ace36600e29032, user.id=90915e1264324f16bc7bae7e46e0c97b) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:27:10.817480 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62f2dab-3ab7-46e6-b7d8-086b735bc137 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:10.817480 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62f2dab-3ab7-46e6-b7d8-086b735bc137 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:10.831042 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 490/977] 10.222.0.81 () {56 vars in 1374 bytes} [Wed Jul 20 22:27:10 2022] GET /identity/v3/role_assignments?user.id=90915e1264324f16bc7bae7e46e0c97b&scope.project.id=385edf2e59e546d090ace36600e29032&role.id=a3e411d61a234343b961e8bf78a55a11 => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:27:10.908506 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3ccc64-0db1-4772-8be8-8dc45dd68366 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:10.935558 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:10.947721 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294467a5-3229-4789-b416-e6aa2a11284c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:10.948645 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294467a5-3229-4789-b416-e6aa2a11284c None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:10.949007 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294467a5-3229-4789-b416-e6aa2a11284c None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:10.949285 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294467a5-3229-4789-b416-e6aa2a11284c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:10.958144 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294467a5-3229-4789-b416-e6aa2a11284c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:10.964041 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294467a5-3229-4789-b416-e6aa2a11284c None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:10.976298 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-294467a5-3229-4789-b416-e6aa2a11284c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 20 22:27:10.976298 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:27:10.976298 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:27:10.976298 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:27:10.976298 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:27:10.976298 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:27:10.976298 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:27:10.976298 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:27:10.976298 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:27:10.976298 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:27:10.976298 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:27:10.976298 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:27:10.976298 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 22:27:10.976298 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 22:27:10.976298 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 22:27:10.976298 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 22:27:10.977283 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:27:10.977283 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:27:10.977283 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 22:27:10.977283 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 22:27:10.977283 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 22:27:10.977283 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 22:27:10.977283 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:27:10.977283 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:27:10.977283 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:27:10.977283 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:27:10.977283 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:27:10.977283 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:27:10.977283 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:27:10.977283 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:27:10.977283 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:27:10.977283 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:27:10.977283 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:27:10.977283 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:27:10.978230 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:27:10.978230 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:27:10.978230 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 22:27:10.978230 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 22:27:10.978230 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 22:27:10.978230 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 22:27:10.978230 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 22:27:10.978230 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 22:27:10.978230 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 20 22:27:10.978230 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:27:10.978230 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 488/978] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Jul 20 22:27:10 2022] GET /identity/v3/users/cinder => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 22:27:10.984119 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62f2dab-3ab7-46e6-b7d8-086b735bc137 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:11.101699 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62f2dab-3ab7-46e6-b7d8-086b735bc137 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:11.136515 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff3ea30-92d3-42ad-b5dc-2993d6edf05c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:11.137201 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff3ea30-92d3-42ad-b5dc-2993d6edf05c None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:11.137488 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff3ea30-92d3-42ad-b5dc-2993d6edf05c None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:11.137773 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff3ea30-92d3-42ad-b5dc-2993d6edf05c None admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:11.138578 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff3ea30-92d3-42ad-b5dc-2993d6edf05c None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:27:11.138892 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff3ea30-92d3-42ad-b5dc-2993d6edf05c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:11.140367 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff3ea30-92d3-42ad-b5dc-2993d6edf05c None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:11.183207 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 491/979] 10.222.0.81 () {56 vars in 1101 bytes} [Wed Jul 20 22:27:10 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:27:11.626740 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426238e8-ec07-4140-9a89-65cc9640dfc7 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:11.627297 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426238e8-ec07-4140-9a89-65cc9640dfc7 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:11.627377 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426238e8-ec07-4140-9a89-65cc9640dfc7 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:11.631707 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 489/980] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:27:11 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:27:11.636825 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b7951b-2c96-4a84-8cb2-83c1e6d90d6e None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:11.637259 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b7951b-2c96-4a84-8cb2-83c1e6d90d6e None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:11.637424 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b7951b-2c96-4a84-8cb2-83c1e6d90d6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:11.654606 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-c6b7951b-2c96-4a84-8cb2-83c1e6d90d6e None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:27:11.655832 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-c6b7951b-2c96-4a84-8cb2-83c1e6d90d6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:27:11.687138 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:11.725376 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 492/981] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:27:11 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 22:27:11.748615 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426238e8-ec07-4140-9a89-65cc9640dfc7 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:11.854335 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426238e8-ec07-4140-9a89-65cc9640dfc7 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:11.908567 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ab1905-96bc-496b-8b3a-baab531878dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:11.909310 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ab1905-96bc-496b-8b3a-baab531878dc None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:11.909592 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ab1905-96bc-496b-8b3a-baab531878dc None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:11.909869 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ab1905-96bc-496b-8b3a-baab531878dc None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:11.910773 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ab1905-96bc-496b-8b3a-baab531878dc None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:11.912227 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ab1905-96bc-496b-8b3a-baab531878dc None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:11.925441 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-98ab1905-96bc-496b-8b3a-baab531878dc None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 22:27:11.925441 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:27:11.925441 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 22:27:11.925441 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 22:27:11.925441 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 22:27:11.925441 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 22:27:11.925441 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 22:27:11.925441 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 22:27:11.925441 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 22:27:11.925441 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:27:11.925441 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 22:27:11.925441 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:27:11.925441 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:27:11.925441 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:27:11.925441 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:27:11.925441 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:27:11.925441 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:27:11.926415 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:27:11.926415 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:27:11.926415 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:27:11.926415 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:27:11.926415 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:27:11.926415 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:27:11.926415 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 22:27:11.926415 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 22:27:11.926415 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 22:27:11.926415 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 22:27:11.926415 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:27:11.926415 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:27:11.926415 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:27:11.926415 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:27:11.926415 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:27:11.926415 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:27:11.927304 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:27:11.927304 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:27:11.927304 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:27:11.927304 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:27:11.927304 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:27:11.927304 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:27:11.927304 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:27:11.927304 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:27:11.927304 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:27:11.927304 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:27:11.927304 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 22:27:11.927304 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 22:27:11.927304 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 22:27:11.927304 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:27:11.929021 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 490/982] 10.222.0.81 () {56 vars in 1126 bytes} [Wed Jul 20 22:27:11 2022] GET /identity/v3/services/object-store => generated 93 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:27:11.932548 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b7951b-2c96-4a84-8cb2-83c1e6d90d6e None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:11.966534 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c203d7cd-66ee-42e3-8b64-b1a149072d7f None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:11.966823 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c203d7cd-66ee-42e3-8b64-b1a149072d7f None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:11.967146 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c203d7cd-66ee-42e3-8b64-b1a149072d7f None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:11.968512 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 491/983] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:27:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 22:27:11.973588 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4faf2c-2f6f-4c0b-a90d-69c775956350 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:11.973881 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4faf2c-2f6f-4c0b-a90d-69c775956350 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:11.974166 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4faf2c-2f6f-4c0b-a90d-69c775956350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:11.991951 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-1c4faf2c-2f6f-4c0b-a90d-69c775956350 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:27:11.993903 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-1c4faf2c-2f6f-4c0b-a90d-69c775956350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:27:12.040791 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b7951b-2c96-4a84-8cb2-83c1e6d90d6e None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:12.041809 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:12.072165 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 492/984] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:27:11 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 22:27:12.079430 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad3f999-1b0a-4187-ab9e-c19989f19e24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:12.080292 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad3f999-1b0a-4187-ab9e-c19989f19e24 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:12.081259 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad3f999-1b0a-4187-ab9e-c19989f19e24 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:12.081702 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad3f999-1b0a-4187-ab9e-c19989f19e24 None admin] PATH_INFO: `/v3/services` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:12.082588 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad3f999-1b0a-4187-ab9e-c19989f19e24 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:27:12.083148 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad3f999-1b0a-4187-ab9e-c19989f19e24 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:12.084692 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad3f999-1b0a-4187-ab9e-c19989f19e24 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:12.095485 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 493/985] 10.222.0.81 () {56 vars in 1122 bytes} [Wed Jul 20 22:27:11 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:27:12.096588 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4faf2c-2f6f-4c0b-a90d-69c775956350 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:12.096724 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad3f999-1b0a-4187-ab9e-c19989f19e24 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:12.129500 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:12.207183 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad3f999-1b0a-4187-ab9e-c19989f19e24 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:12.209466 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4faf2c-2f6f-4c0b-a90d-69c775956350 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:12.243606 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9d051b-0bd7-4e0e-bfeb-df77c4f05319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:12.244104 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e055cd0-6977-48f9-85b5-15cb55d6952d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:12.244321 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9d051b-0bd7-4e0e-bfeb-df77c4f05319 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:12.244573 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9d051b-0bd7-4e0e-bfeb-df77c4f05319 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:12.244774 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e055cd0-6977-48f9-85b5-15cb55d6952d None admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:12.244876 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9d051b-0bd7-4e0e-bfeb-df77c4f05319 None admin] PATH_INFO: `/v3/services` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:12.245061 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e055cd0-6977-48f9-85b5-15cb55d6952d None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:12.245337 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e055cd0-6977-48f9-85b5-15cb55d6952d None admin] PATH_INFO: `/v3/services` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:12.245545 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9d051b-0bd7-4e0e-bfeb-df77c4f05319 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:27:12.245858 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9d051b-0bd7-4e0e-bfeb-df77c4f05319 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:12.246053 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e055cd0-6977-48f9-85b5-15cb55d6952d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:12.247268 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9d051b-0bd7-4e0e-bfeb-df77c4f05319 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:12.247430 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e055cd0-6977-48f9-85b5-15cb55d6952d None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:12.253254 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 493/986] 10.222.0.81 () {56 vars in 1122 bytes} [Wed Jul 20 22:27:12 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 22:27:12.259219 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9d051b-0bd7-4e0e-bfeb-df77c4f05319 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:12.301876 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:12.315876 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 494/987] 10.222.0.81 () {60 vars in 1141 bytes} [Wed Jul 20 22:27:12 2022] POST /identity/v3/services => generated 242 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 22:27:12.373165 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9d051b-0bd7-4e0e-bfeb-df77c4f05319 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:12.417968 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfee202-db93-4887-9d76-34f5654c9672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:12.418656 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfee202-db93-4887-9d76-34f5654c9672 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:12.418944 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfee202-db93-4887-9d76-34f5654c9672 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:12.419246 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfee202-db93-4887-9d76-34f5654c9672 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:12.420059 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfee202-db93-4887-9d76-34f5654c9672 None admin] RBAC: Adding query filter params (interface=admin, service_id=29c9b2f63b364c7abd152381e6e9c44f, region_id=RegionOne) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:27:12.420382 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfee202-db93-4887-9d76-34f5654c9672 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:12.421773 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfee202-db93-4887-9d76-34f5654c9672 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:12.432427 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 494/988] 10.222.0.81 () {56 vars in 1249 bytes} [Wed Jul 20 22:27:12 2022] GET /identity/v3/endpoints?service_id=29c9b2f63b364c7abd152381e6e9c44f&interface=admin®ion_id=RegionOne => generated 193 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 22:27:12.433726 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e055cd0-6977-48f9-85b5-15cb55d6952d None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:12.532260 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:12.541866 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e055cd0-6977-48f9-85b5-15cb55d6952d None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:12.577053 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01de42b2-79b1-4e36-8436-8f24015095bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:12.577727 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01de42b2-79b1-4e36-8436-8f24015095bc None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:12.578013 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01de42b2-79b1-4e36-8436-8f24015095bc None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:12.578357 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01de42b2-79b1-4e36-8436-8f24015095bc None admin] PATH_INFO: `/v3/services` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:12.579057 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01de42b2-79b1-4e36-8436-8f24015095bc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:12.580428 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01de42b2-79b1-4e36-8436-8f24015095bc None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:12.586878 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 495/989] 10.222.0.81 () {56 vars in 1087 bytes} [Wed Jul 20 22:27:12 2022] GET /identity/v3/services => generated 1727 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:27:12.947466 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b667b98-14c0-4bea-9c4d-8cdc36224266 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:12.948019 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b667b98-14c0-4bea-9c4d-8cdc36224266 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:12.948019 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b667b98-14c0-4bea-9c4d-8cdc36224266 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:12.949521 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 495/990] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:27:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:27:12.954604 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2dc243-af75-4d56-8f00-38fc3edab519 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:12.954999 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2dc243-af75-4d56-8f00-38fc3edab519 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:12.955289 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2dc243-af75-4d56-8f00-38fc3edab519 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:12.972400 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-6f2dc243-af75-4d56-8f00-38fc3edab519 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:27:12.973582 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-6f2dc243-af75-4d56-8f00-38fc3edab519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:27:13.043743 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 496/991] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:27:12 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 22:27:13.065738 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b667b98-14c0-4bea-9c4d-8cdc36224266 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:13.167338 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:13.171895 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b667b98-14c0-4bea-9c4d-8cdc36224266 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:13.212093 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f973244e-cefa-424e-b9d9-2e1e446aeaff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:13.212843 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f973244e-cefa-424e-b9d9-2e1e446aeaff None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:13.213126 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f973244e-cefa-424e-b9d9-2e1e446aeaff None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:13.213402 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f973244e-cefa-424e-b9d9-2e1e446aeaff None admin] PATH_INFO: `/v3/services/image` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:13.214047 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f973244e-cefa-424e-b9d9-2e1e446aeaff None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:13.215442 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f973244e-cefa-424e-b9d9-2e1e446aeaff None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:13.228002 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-f973244e-cefa-424e-b9d9-2e1e446aeaff None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 20 22:27:13.228002 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:27:13.228002 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 22:27:13.228002 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 22:27:13.228002 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 22:27:13.228002 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 22:27:13.228002 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 22:27:13.228002 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 22:27:13.228002 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 20 22:27:13.228002 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:27:13.228002 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 22:27:13.228002 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:27:13.228002 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:27:13.228002 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:27:13.228002 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:27:13.228002 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:27:13.228002 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:27:13.229012 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:27:13.229012 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:27:13.229012 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:27:13.229012 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:27:13.229012 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:27:13.229012 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:27:13.229012 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 22:27:13.229012 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 22:27:13.229012 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 22:27:13.229012 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 22:27:13.229012 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:27:13.229012 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:27:13.229012 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:27:13.229012 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:27:13.229012 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:27:13.229012 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:27:13.229848 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:27:13.229848 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:27:13.229848 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:27:13.229848 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:27:13.229848 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:27:13.229848 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:27:13.229848 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:27:13.229848 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:27:13.229848 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:27:13.229848 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:27:13.229848 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 22:27:13.229848 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 22:27:13.229848 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 20 22:27:13.229848 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:27:13.234180 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 496/992] 10.222.0.81 () {56 vars in 1105 bytes} [Wed Jul 20 22:27:13 2022] GET /identity/v3/services/image => generated 86 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:27:13.235508 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2dc243-af75-4d56-8f00-38fc3edab519 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:13.322071 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:13.338540 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2dc243-af75-4d56-8f00-38fc3edab519 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:13.373636 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9198fa-a696-428f-9bfe-e06dd49e6985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:13.374310 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9198fa-a696-428f-9bfe-e06dd49e6985 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:13.374598 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9198fa-a696-428f-9bfe-e06dd49e6985 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:13.374898 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9198fa-a696-428f-9bfe-e06dd49e6985 None admin] PATH_INFO: `/v3/services` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:13.375637 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9198fa-a696-428f-9bfe-e06dd49e6985 None admin] RBAC: Adding query filter params (name=image) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:27:13.375948 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9198fa-a696-428f-9bfe-e06dd49e6985 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:13.377313 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9198fa-a696-428f-9bfe-e06dd49e6985 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:13.386814 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 497/993] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Jul 20 22:27:13 2022] GET /identity/v3/services?name=image => generated 122 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:27:13.387826 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f973244e-cefa-424e-b9d9-2e1e446aeaff None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:13.496758 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f973244e-cefa-424e-b9d9-2e1e446aeaff None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:13.529593 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e621e85b-7edc-42b8-b9b7-b9c09ca9e862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:13.530264 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e621e85b-7edc-42b8-b9b7-b9c09ca9e862 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:13.530549 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e621e85b-7edc-42b8-b9b7-b9c09ca9e862 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:13.530826 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e621e85b-7edc-42b8-b9b7-b9c09ca9e862 None admin] PATH_INFO: `/v3/services` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:13.532130 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e621e85b-7edc-42b8-b9b7-b9c09ca9e862 None admin] RBAC: Adding query filter params (type=image) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:27:13.532440 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e621e85b-7edc-42b8-b9b7-b9c09ca9e862 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:13.533792 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e621e85b-7edc-42b8-b9b7-b9c09ca9e862 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:13.538661 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 497/994] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Jul 20 22:27:13 2022] GET /identity/v3/services?type=image => generated 122 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 22:27:14.382979 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac5e5d6-7ec9-45b0-a736-ccd209bb82b1 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:14.383553 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac5e5d6-7ec9-45b0-a736-ccd209bb82b1 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:14.383553 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac5e5d6-7ec9-45b0-a736-ccd209bb82b1 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:14.385068 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 498/995] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:27:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:27:14.389969 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcba6d4-4291-4ad9-8682-0814c144fbf6 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:14.390345 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcba6d4-4291-4ad9-8682-0814c144fbf6 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:14.391196 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcba6d4-4291-4ad9-8682-0814c144fbf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:14.407709 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-edcba6d4-4291-4ad9-8682-0814c144fbf6 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:27:14.408907 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-edcba6d4-4291-4ad9-8682-0814c144fbf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:27:14.431952 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:14.479225 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 498/996] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:27:14 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 22:27:14.497896 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac5e5d6-7ec9-45b0-a736-ccd209bb82b1 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:14.602021 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac5e5d6-7ec9-45b0-a736-ccd209bb82b1 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:14.627506 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4ab447-534f-4188-a419-9e7157ea55e2 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:14.627834 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4ab447-534f-4188-a419-9e7157ea55e2 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:14.628071 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4ab447-534f-4188-a419-9e7157ea55e2 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:14.629440 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 499/997] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:27:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 22:27:14.637791 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acb4dfa-aa08-4186-9c59-86427840e4e6 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:14.638082 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acb4dfa-aa08-4186-9c59-86427840e4e6 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:14.638363 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acb4dfa-aa08-4186-9c59-86427840e4e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:14.639953 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700ee5b7-eb8c-4392-9786-7852075c2c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:14.641002 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700ee5b7-eb8c-4392-9786-7852075c2c22 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:14.641474 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700ee5b7-eb8c-4392-9786-7852075c2c22 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:14.641903 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700ee5b7-eb8c-4392-9786-7852075c2c22 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:14.642726 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700ee5b7-eb8c-4392-9786-7852075c2c22 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:14.644353 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700ee5b7-eb8c-4392-9786-7852075c2c22 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:14.656885 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-3acb4dfa-aa08-4186-9c59-86427840e4e6 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:27:14.657822 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-700ee5b7-eb8c-4392-9786-7852075c2c22 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 20 22:27:14.657822 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:27:14.657822 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 22:27:14.657822 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 22:27:14.657822 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 22:27:14.657822 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 22:27:14.657822 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 22:27:14.657822 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 22:27:14.657822 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 20 22:27:14.657822 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:27:14.657822 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 22:27:14.657822 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:27:14.657822 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:27:14.657822 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:27:14.657822 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:27:14.657822 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:27:14.657822 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:27:14.658776 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:27:14.658776 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:27:14.658776 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:27:14.658776 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:27:14.658776 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:27:14.658776 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:27:14.658776 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 22:27:14.658776 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 22:27:14.658776 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 22:27:14.658776 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 22:27:14.658776 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:27:14.658776 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:27:14.658776 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:27:14.658776 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:27:14.658776 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:27:14.658776 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:27:14.659973 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:27:14.659973 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:27:14.659973 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:27:14.659973 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:27:14.659973 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:27:14.659973 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:27:14.659973 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:27:14.659973 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:27:14.659973 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:27:14.659973 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:27:14.659973 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 22:27:14.659973 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 22:27:14.659973 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 20 22:27:14.659973 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:27:14.660968 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-3acb4dfa-aa08-4186-9c59-86427840e4e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:27:14.661047 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 499/998] 10.222.0.81 () {56 vars in 1111 bytes} [Wed Jul 20 22:27:14 2022] GET /identity/v3/services/compute => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:27:14.665145 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700ee5b7-eb8c-4392-9786-7852075c2c22 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:14.683216 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:14.734762 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 500/999] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:27:14 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 22:27:14.757591 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acb4dfa-aa08-4186-9c59-86427840e4e6 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:14.775128 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700ee5b7-eb8c-4392-9786-7852075c2c22 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:14.811079 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673ea051-2e01-4c10-ad28-f51d88316c80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:14.811768 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673ea051-2e01-4c10-ad28-f51d88316c80 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:14.812039 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673ea051-2e01-4c10-ad28-f51d88316c80 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:14.812352 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673ea051-2e01-4c10-ad28-f51d88316c80 None admin] PATH_INFO: `/v3/services` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:14.813112 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673ea051-2e01-4c10-ad28-f51d88316c80 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:27:14.813423 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673ea051-2e01-4c10-ad28-f51d88316c80 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:14.814791 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673ea051-2e01-4c10-ad28-f51d88316c80 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:14.819671 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 500/1000] 10.222.0.81 () {56 vars in 1112 bytes} [Wed Jul 20 22:27:14 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 22:27:14.825511 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673ea051-2e01-4c10-ad28-f51d88316c80 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:14.858297 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:14.871749 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acb4dfa-aa08-4186-9c59-86427840e4e6 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:14.902323 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:14.908817 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e893c30e-d2b3-4d4a-88a5-16251a135395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:14.909647 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e893c30e-d2b3-4d4a-88a5-16251a135395 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:14.909934 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e893c30e-d2b3-4d4a-88a5-16251a135395 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:14.910454 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e893c30e-d2b3-4d4a-88a5-16251a135395 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:14.916995 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e893c30e-d2b3-4d4a-88a5-16251a135395 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:14.916995 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e893c30e-d2b3-4d4a-88a5-16251a135395 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:14.925455 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-e893c30e-d2b3-4d4a-88a5-16251a135395 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 22:27:14.925455 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:27:14.925455 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 22:27:14.925455 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 22:27:14.925455 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 22:27:14.925455 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 22:27:14.925455 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 22:27:14.925455 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 22:27:14.925455 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 22:27:14.925455 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:27:14.925455 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 22:27:14.925455 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:27:14.925455 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:27:14.925455 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:27:14.925455 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:27:14.925455 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:27:14.925455 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:27:14.926457 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:27:14.926457 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:27:14.926457 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:27:14.926457 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:27:14.926457 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:27:14.926457 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:27:14.926457 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 22:27:14.926457 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 22:27:14.926457 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 22:27:14.926457 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 22:27:14.926457 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:27:14.926457 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:27:14.926457 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:27:14.926457 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:27:14.926457 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:27:14.926457 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:27:14.927962 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:27:14.927962 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:27:14.927962 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:27:14.927962 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:27:14.927962 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:27:14.927962 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:27:14.927962 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:27:14.927962 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:27:14.927962 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:27:14.927962 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:27:14.927962 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 22:27:14.927962 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 22:27:14.927962 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 22:27:14.927962 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:27:14.928898 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673ea051-2e01-4c10-ad28-f51d88316c80 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:14.931816 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 501/1001] 10.222.0.81 () {56 vars in 1126 bytes} [Wed Jul 20 22:27:14 2022] GET /identity/v3/services/object-store => generated 93 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 22:27:14.932653 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e893c30e-d2b3-4d4a-88a5-16251a135395 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:14.964405 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e8b3fe-af24-4d58-b4cc-c06a5e4542ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:14.965068 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e8b3fe-af24-4d58-b4cc-c06a5e4542ad None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:14.965370 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e8b3fe-af24-4d58-b4cc-c06a5e4542ad None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:14.965631 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e8b3fe-af24-4d58-b4cc-c06a5e4542ad None admin] PATH_INFO: `/v3/services` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:14.966332 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e8b3fe-af24-4d58-b4cc-c06a5e4542ad None admin] RBAC: Adding query filter params (type=compute) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:27:14.966640 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e8b3fe-af24-4d58-b4cc-c06a5e4542ad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:14.968041 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e8b3fe-af24-4d58-b4cc-c06a5e4542ad None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:14.974026 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 501/1002] 10.222.0.81 () {56 vars in 1112 bytes} [Wed Jul 20 22:27:14 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 22:27:14.980123 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e8b3fe-af24-4d58-b4cc-c06a5e4542ad None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:15.042831 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e893c30e-d2b3-4d4a-88a5-16251a135395 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:15.078859 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd920385-d7ba-4f34-a2dc-9382763b72e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:15.079565 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd920385-d7ba-4f34-a2dc-9382763b72e9 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:15.079856 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd920385-d7ba-4f34-a2dc-9382763b72e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:15.080136 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd920385-d7ba-4f34-a2dc-9382763b72e9 None admin] PATH_INFO: `/v3/services` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:15.080842 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd920385-d7ba-4f34-a2dc-9382763b72e9 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:27:15.081153 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd920385-d7ba-4f34-a2dc-9382763b72e9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:15.082514 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd920385-d7ba-4f34-a2dc-9382763b72e9 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:15.086492 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e8b3fe-af24-4d58-b4cc-c06a5e4542ad None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:15.087461 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 502/1003] 10.222.0.81 () {56 vars in 1122 bytes} [Wed Jul 20 22:27:14 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:27:15.093173 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd920385-d7ba-4f34-a2dc-9382763b72e9 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:15.111921 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:15.128602 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb304fe-335d-4368-b120-b00bde91d0e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:15.129285 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb304fe-335d-4368-b120-b00bde91d0e3 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:15.129571 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb304fe-335d-4368-b120-b00bde91d0e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:15.129849 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb304fe-335d-4368-b120-b00bde91d0e3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:15.130673 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb304fe-335d-4368-b120-b00bde91d0e3 None admin] RBAC: Adding query filter params (interface=public, service_id=98b87f7951e64c84b5cfcd7039f124ab, region_id=RegionOne) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:27:15.130952 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb304fe-335d-4368-b120-b00bde91d0e3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:15.132347 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb304fe-335d-4368-b120-b00bde91d0e3 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:15.142484 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 502/1004] 10.222.0.81 () {56 vars in 1251 bytes} [Wed Jul 20 22:27:14 2022] GET /identity/v3/endpoints?service_id=98b87f7951e64c84b5cfcd7039f124ab&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 22:27:15.143583 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb304fe-335d-4368-b120-b00bde91d0e3 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:15.147232 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:15.210605 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd920385-d7ba-4f34-a2dc-9382763b72e9 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:15.252378 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb304fe-335d-4368-b120-b00bde91d0e3 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:15.252569 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b6a789-d6b8-4cb1-b854-d9afda2d5275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:15.253093 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b6a789-d6b8-4cb1-b854-d9afda2d5275 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:15.253381 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b6a789-d6b8-4cb1-b854-d9afda2d5275 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:15.253661 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b6a789-d6b8-4cb1-b854-d9afda2d5275 None admin] PATH_INFO: `/v3/services` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:15.254364 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b6a789-d6b8-4cb1-b854-d9afda2d5275 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:27:15.254679 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b6a789-d6b8-4cb1-b854-d9afda2d5275 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:15.256075 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b6a789-d6b8-4cb1-b854-d9afda2d5275 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:15.261280 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 503/1005] 10.222.0.81 () {56 vars in 1122 bytes} [Wed Jul 20 22:27:15 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 22:27:15.267828 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b6a789-d6b8-4cb1-b854-d9afda2d5275 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:15.296399 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b82ba4-8ec6-445c-8fdb-4a8065cf78a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:15.297071 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b82ba4-8ec6-445c-8fdb-4a8065cf78a9 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:15.297471 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b82ba4-8ec6-445c-8fdb-4a8065cf78a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:15.297650 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b82ba4-8ec6-445c-8fdb-4a8065cf78a9 None admin] PATH_INFO: `/v3/services` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:15.298311 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b82ba4-8ec6-445c-8fdb-4a8065cf78a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:15.299725 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b82ba4-8ec6-445c-8fdb-4a8065cf78a9 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:15.306297 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 503/1006] 10.222.0.81 () {56 vars in 1087 bytes} [Wed Jul 20 22:27:15 2022] GET /identity/v3/services => generated 1727 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 22:27:15.369362 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:15.389617 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b6a789-d6b8-4cb1-b854-d9afda2d5275 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:15.426461 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d39f0a1-5b64-4a86-beb9-93459fb29ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:15.427274 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d39f0a1-5b64-4a86-beb9-93459fb29ff9 None admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:15.427475 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d39f0a1-5b64-4a86-beb9-93459fb29ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:15.427755 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d39f0a1-5b64-4a86-beb9-93459fb29ff9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:15.428518 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d39f0a1-5b64-4a86-beb9-93459fb29ff9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:15.429902 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d39f0a1-5b64-4a86-beb9-93459fb29ff9 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:15.538292 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 504/1007] 10.222.0.81 () {60 vars in 1144 bytes} [Wed Jul 20 22:27:15 2022] POST /identity/v3/endpoints => generated 325 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 22:27:15.600830 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dd08a6-a1c7-429b-91b9-74ed07d2fd08 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:15.601392 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dd08a6-a1c7-429b-91b9-74ed07d2fd08 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:15.601392 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dd08a6-a1c7-429b-91b9-74ed07d2fd08 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:15.602923 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 504/1008] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:27:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:27:15.608446 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309507ef-ffaa-4876-827a-8a5685fe9807 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:15.608740 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309507ef-ffaa-4876-827a-8a5685fe9807 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:15.609072 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309507ef-ffaa-4876-827a-8a5685fe9807 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:15.626949 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-309507ef-ffaa-4876-827a-8a5685fe9807 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:27:15.628213 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-309507ef-ffaa-4876-827a-8a5685fe9807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:27:15.709981 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 505/1009] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:27:15 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 22:27:15.733544 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dd08a6-a1c7-429b-91b9-74ed07d2fd08 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:15.804788 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:15.840230 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dd08a6-a1c7-429b-91b9-74ed07d2fd08 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:15.876839 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c52ca3d-cf59-4bcb-9b79-582888c2a32c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:15.877431 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c52ca3d-cf59-4bcb-9b79-582888c2a32c None admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:15.877721 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c52ca3d-cf59-4bcb-9b79-582888c2a32c None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:15.878004 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c52ca3d-cf59-4bcb-9b79-582888c2a32c None admin] PATH_INFO: `/v3/services` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:15.878730 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c52ca3d-cf59-4bcb-9b79-582888c2a32c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:15.880244 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c52ca3d-cf59-4bcb-9b79-582888c2a32c None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:15.978221 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 505/1010] 10.222.0.81 () {60 vars in 1141 bytes} [Wed Jul 20 22:27:15 2022] POST /identity/v3/services => generated 242 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:27:17.356985 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6875f7-4a0a-4441-8965-7000191780d7 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:17.357495 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6875f7-4a0a-4441-8965-7000191780d7 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:17.357603 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6875f7-4a0a-4441-8965-7000191780d7 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:17.359138 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 506/1011] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:27:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:27:17.364111 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bad99e-6a50-4521-ba3f-c7ebbedf7ec5 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:17.364507 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bad99e-6a50-4521-ba3f-c7ebbedf7ec5 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:17.364711 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bad99e-6a50-4521-ba3f-c7ebbedf7ec5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:17.382654 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-50bad99e-6a50-4521-ba3f-c7ebbedf7ec5 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:27:17.383879 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-50bad99e-6a50-4521-ba3f-c7ebbedf7ec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:27:17.457348 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 506/1012] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:27:17 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 22:27:17.480223 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6875f7-4a0a-4441-8965-7000191780d7 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:17.545972 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:17.581547 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6875f7-4a0a-4441-8965-7000191780d7 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:17.600806 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3f79e8-a96b-4211-a02e-7ac67571a7bf None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:17.601090 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3f79e8-a96b-4211-a02e-7ac67571a7bf None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:17.602583 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3f79e8-a96b-4211-a02e-7ac67571a7bf None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:17.602863 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 507/1013] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:27:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 22:27:17.607865 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f458176-e1c1-4814-be2b-5b423eb68822 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:17.608147 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f458176-e1c1-4814-be2b-5b423eb68822 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:17.608425 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f458176-e1c1-4814-be2b-5b423eb68822 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:17.619133 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e104b897-455b-4fe1-81d6-93e2cd603cc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:17.619952 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e104b897-455b-4fe1-81d6-93e2cd603cc1 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:17.620236 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e104b897-455b-4fe1-81d6-93e2cd603cc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:17.620513 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e104b897-455b-4fe1-81d6-93e2cd603cc1 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:17.621164 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e104b897-455b-4fe1-81d6-93e2cd603cc1 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:17.622555 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e104b897-455b-4fe1-81d6-93e2cd603cc1 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:17.629475 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-8f458176-e1c1-4814-be2b-5b423eb68822 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:27:17.630671 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-8f458176-e1c1-4814-be2b-5b423eb68822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:27:17.636089 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-e104b897-455b-4fe1-81d6-93e2cd603cc1 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 20 22:27:17.636089 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:27:17.636089 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 22:27:17.636089 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 22:27:17.636089 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 22:27:17.636089 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 22:27:17.636089 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 22:27:17.636089 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 22:27:17.636089 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 20 22:27:17.636089 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:27:17.636089 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 22:27:17.636089 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:27:17.636089 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:27:17.636089 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:27:17.636089 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:27:17.636089 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:27:17.636089 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:27:17.637028 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:27:17.637028 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:27:17.637028 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:27:17.637028 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:27:17.637028 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:27:17.637028 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:27:17.637028 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 22:27:17.637028 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 22:27:17.637028 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 22:27:17.637028 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 22:27:17.637028 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:27:17.637028 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:27:17.637028 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:27:17.637028 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:27:17.637028 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:27:17.637028 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:27:17.637867 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:27:17.637867 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:27:17.637867 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:27:17.637867 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:27:17.637867 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:27:17.637867 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:27:17.637867 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:27:17.637867 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:27:17.637867 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:27:17.637867 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:27:17.637867 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 22:27:17.637867 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 22:27:17.637867 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 20 22:27:17.637867 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:27:17.642774 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 507/1014] 10.222.0.81 () {56 vars in 1111 bytes} [Wed Jul 20 22:27:17 2022] GET /identity/v3/services/compute => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:27:17.643851 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e104b897-455b-4fe1-81d6-93e2cd603cc1 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:17.694756 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:17.725051 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 508/1015] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:27:17 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 22:27:17.743909 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f458176-e1c1-4814-be2b-5b423eb68822 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:17.749193 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e104b897-455b-4fe1-81d6-93e2cd603cc1 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:17.779360 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:17.785818 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80baee1f-0713-41d9-a529-69cb7a93221b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:17.786499 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80baee1f-0713-41d9-a529-69cb7a93221b None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:17.786789 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80baee1f-0713-41d9-a529-69cb7a93221b None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:17.787413 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80baee1f-0713-41d9-a529-69cb7a93221b None admin] PATH_INFO: `/v3/services` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:17.788118 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80baee1f-0713-41d9-a529-69cb7a93221b None admin] RBAC: Adding query filter params (name=compute) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:27:17.788439 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80baee1f-0713-41d9-a529-69cb7a93221b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:17.789801 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80baee1f-0713-41d9-a529-69cb7a93221b None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:17.794697 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 508/1016] 10.222.0.81 () {56 vars in 1112 bytes} [Wed Jul 20 22:27:17 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:27:17.801059 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80baee1f-0713-41d9-a529-69cb7a93221b None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:17.875713 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f458176-e1c1-4814-be2b-5b423eb68822 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:17.911461 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80baee1f-0713-41d9-a529-69cb7a93221b None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:17.933174 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8d82f4-a155-4ca1-925e-d6ae19e21de1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:17.934359 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8d82f4-a155-4ca1-925e-d6ae19e21de1 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:17.934803 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8d82f4-a155-4ca1-925e-d6ae19e21de1 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:17.935108 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8d82f4-a155-4ca1-925e-d6ae19e21de1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:17.938284 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8d82f4-a155-4ca1-925e-d6ae19e21de1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:17.940081 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8d82f4-a155-4ca1-925e-d6ae19e21de1 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:17.946532 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 509/1017] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Jul 20 22:27:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:27:17.947482 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8d82f4-a155-4ca1-925e-d6ae19e21de1 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:17.956849 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab5e33b-1a62-4c83-9fdd-15b62885a85b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:17.957537 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab5e33b-1a62-4c83-9fdd-15b62885a85b None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:17.957826 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab5e33b-1a62-4c83-9fdd-15b62885a85b None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:17.958105 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab5e33b-1a62-4c83-9fdd-15b62885a85b None admin] PATH_INFO: `/v3/services` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:17.959213 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab5e33b-1a62-4c83-9fdd-15b62885a85b None admin] RBAC: Adding query filter params (type=compute) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:27:17.959669 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab5e33b-1a62-4c83-9fdd-15b62885a85b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:17.961913 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab5e33b-1a62-4c83-9fdd-15b62885a85b None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:17.970447 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 509/1018] 10.222.0.81 () {56 vars in 1112 bytes} [Wed Jul 20 22:27:17 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 22:27:17.976072 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab5e33b-1a62-4c83-9fdd-15b62885a85b None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:18.014945 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:18.043096 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:18.067671 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8d82f4-a155-4ca1-925e-d6ae19e21de1 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:18.085195 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab5e33b-1a62-4c83-9fdd-15b62885a85b None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:18.100711 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c91a4ff-00f7-4b81-88bd-27cfca05753a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:18.101376 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c91a4ff-00f7-4b81-88bd-27cfca05753a None admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:18.101669 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c91a4ff-00f7-4b81-88bd-27cfca05753a None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:18.101956 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c91a4ff-00f7-4b81-88bd-27cfca05753a None admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:18.102666 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c91a4ff-00f7-4b81-88bd-27cfca05753a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:18.104133 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c91a4ff-00f7-4b81-88bd-27cfca05753a None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:18.118563 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1b4d82-831d-43a7-9628-6e9e71d4d576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:18.119253 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1b4d82-831d-43a7-9628-6e9e71d4d576 None admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:18.119540 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1b4d82-831d-43a7-9628-6e9e71d4d576 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:18.119820 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1b4d82-831d-43a7-9628-6e9e71d4d576 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:18.120551 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1b4d82-831d-43a7-9628-6e9e71d4d576 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:18.121901 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1b4d82-831d-43a7-9628-6e9e71d4d576 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:18.184237 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 510/1019] 10.222.0.81 () {60 vars in 1141 bytes} [Wed Jul 20 22:27:17 2022] POST /identity/v3/projects => generated 311 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:27:18.185558 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b371b4af-bc2e-4644-8fab-38e6f1a85aaf None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:18.185876 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b371b4af-bc2e-4644-8fab-38e6f1a85aaf None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:18.186159 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b371b4af-bc2e-4644-8fab-38e6f1a85aaf None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:18.188213 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 511/1020] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:27:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:27:18.194338 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e99cdb-9f6c-422d-a889-34b9f33f2b38 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:18.194867 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e99cdb-9f6c-422d-a889-34b9f33f2b38 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:18.195319 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e99cdb-9f6c-422d-a889-34b9f33f2b38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:18.196450 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 510/1021] 10.222.0.81 () {60 vars in 1144 bytes} [Wed Jul 20 22:27:17 2022] POST /identity/v3/endpoints => generated 334 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 22:27:18.221866 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-62e99cdb-9f6c-422d-a889-34b9f33f2b38 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:27:18.223569 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-62e99cdb-9f6c-422d-a889-34b9f33f2b38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:27:18.317688 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 512/1022] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:27:18 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 22:27:18.341456 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1b4d82-831d-43a7-9628-6e9e71d4d576 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:18.451658 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1b4d82-831d-43a7-9628-6e9e71d4d576 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:18.467607 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:18.488319 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b4c741-c0f8-4b1b-a6ba-0fbe01e49d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:18.489349 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b4c741-c0f8-4b1b-a6ba-0fbe01e49d61 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:18.489814 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b4c741-c0f8-4b1b-a6ba-0fbe01e49d61 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:18.490090 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b4c741-c0f8-4b1b-a6ba-0fbe01e49d61 None admin] PATH_INFO: `/v3/services/image` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:18.490738 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b4c741-c0f8-4b1b-a6ba-0fbe01e49d61 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:18.492137 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b4c741-c0f8-4b1b-a6ba-0fbe01e49d61 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:18.504907 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-29b4c741-c0f8-4b1b-a6ba-0fbe01e49d61 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 20 22:27:18.504907 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:27:18.504907 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 22:27:18.504907 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 22:27:18.504907 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 22:27:18.504907 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 22:27:18.504907 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 22:27:18.504907 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 22:27:18.504907 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 20 22:27:18.504907 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:27:18.504907 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 22:27:18.504907 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:27:18.504907 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:27:18.504907 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:27:18.504907 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:27:18.504907 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:27:18.504907 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:27:18.505949 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:27:18.505949 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:27:18.505949 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:27:18.505949 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:27:18.505949 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:27:18.505949 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:27:18.505949 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 22:27:18.505949 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 22:27:18.505949 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 22:27:18.505949 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 22:27:18.505949 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:27:18.505949 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:27:18.505949 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:27:18.505949 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:27:18.505949 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:27:18.505949 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:27:18.506824 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:27:18.506824 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:27:18.506824 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:27:18.506824 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:27:18.506824 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:27:18.506824 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:27:18.506824 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:27:18.506824 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:27:18.506824 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:27:18.506824 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:27:18.506824 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 22:27:18.506824 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 22:27:18.506824 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 20 22:27:18.506824 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:27:18.506824 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 511/1023] 10.222.0.81 () {56 vars in 1105 bytes} [Wed Jul 20 22:27:18 2022] GET /identity/v3/services/image => generated 86 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:27:18.513203 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e99cdb-9f6c-422d-a889-34b9f33f2b38 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:18.547533 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:18.620884 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e99cdb-9f6c-422d-a889-34b9f33f2b38 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:18.656592 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc2c966-b147-4be4-bd73-564246b27b7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:18.657270 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc2c966-b147-4be4-bd73-564246b27b7c None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:18.657562 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc2c966-b147-4be4-bd73-564246b27b7c None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:18.657847 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc2c966-b147-4be4-bd73-564246b27b7c None admin] PATH_INFO: `/v3/services` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:18.658559 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc2c966-b147-4be4-bd73-564246b27b7c None admin] RBAC: Adding query filter params (name=image) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:27:18.658871 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc2c966-b147-4be4-bd73-564246b27b7c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:18.660534 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc2c966-b147-4be4-bd73-564246b27b7c None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:18.665757 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 513/1024] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Jul 20 22:27:18 2022] GET /identity/v3/services?name=image => generated 122 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:27:18.671393 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b4c741-c0f8-4b1b-a6ba-0fbe01e49d61 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:18.775559 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b4c741-c0f8-4b1b-a6ba-0fbe01e49d61 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:18.810865 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bd3aea-2839-4e1e-a0c8-98e575794e1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:18.811568 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bd3aea-2839-4e1e-a0c8-98e575794e1b None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:18.811881 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bd3aea-2839-4e1e-a0c8-98e575794e1b None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:18.812163 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bd3aea-2839-4e1e-a0c8-98e575794e1b None admin] PATH_INFO: `/v3/services` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:18.812872 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bd3aea-2839-4e1e-a0c8-98e575794e1b None admin] RBAC: Adding query filter params (type=image) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:27:18.813183 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bd3aea-2839-4e1e-a0c8-98e575794e1b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:18.814546 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bd3aea-2839-4e1e-a0c8-98e575794e1b None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:18.824594 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 512/1025] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Jul 20 22:27:18 2022] GET /identity/v3/services?type=image => generated 350 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:27:18.826155 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc2c966-b147-4be4-bd73-564246b27b7c None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:18.922512 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:18.929736 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc2c966-b147-4be4-bd73-564246b27b7c None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:18.963809 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b734e3-768d-4308-acd9-bc1291e8862e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:18.964477 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b734e3-768d-4308-acd9-bc1291e8862e None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:18.964764 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b734e3-768d-4308-acd9-bc1291e8862e None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:18.965044 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b734e3-768d-4308-acd9-bc1291e8862e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:18.965827 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b734e3-768d-4308-acd9-bc1291e8862e None admin] RBAC: Adding query filter params (interface=public, service_id=26a7ff7779794841afc08f6c953a9c22, region_id=RegionOne) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:27:18.966143 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b734e3-768d-4308-acd9-bc1291e8862e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:18.967548 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b734e3-768d-4308-acd9-bc1291e8862e None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:18.977562 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 514/1026] 10.222.0.81 () {56 vars in 1251 bytes} [Wed Jul 20 22:27:18 2022] GET /identity/v3/endpoints?service_id=26a7ff7779794841afc08f6c953a9c22&interface=public®ion_id=RegionOne => generated 194 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:27:18.978766 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bd3aea-2839-4e1e-a0c8-98e575794e1b None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:19.023441 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:19.079997 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bd3aea-2839-4e1e-a0c8-98e575794e1b None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:19.113642 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d020ec06-aef0-4626-8dd9-9abb575ef2b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:19.114308 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d020ec06-aef0-4626-8dd9-9abb575ef2b2 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:19.114594 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d020ec06-aef0-4626-8dd9-9abb575ef2b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:19.114873 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d020ec06-aef0-4626-8dd9-9abb575ef2b2 None admin] PATH_INFO: `/v3/services` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:19.115580 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d020ec06-aef0-4626-8dd9-9abb575ef2b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:19.116942 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d020ec06-aef0-4626-8dd9-9abb575ef2b2 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:19.123653 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 513/1027] 10.222.0.81 () {56 vars in 1087 bytes} [Wed Jul 20 22:27:18 2022] GET /identity/v3/services => generated 1957 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:27:20.308259 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b10901-6677-42bf-a96e-7a396f4f8f5b None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:20.308776 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b10901-6677-42bf-a96e-7a396f4f8f5b None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:20.308854 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b10901-6677-42bf-a96e-7a396f4f8f5b None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:20.310347 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 515/1028] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:27:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:27:20.316172 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd57c678-c0fa-4cea-8d2d-e318458a5ac7 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:20.316599 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd57c678-c0fa-4cea-8d2d-e318458a5ac7 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:20.316819 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd57c678-c0fa-4cea-8d2d-e318458a5ac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:20.334000 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-cd57c678-c0fa-4cea-8d2d-e318458a5ac7 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:27:20.335235 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-cd57c678-c0fa-4cea-8d2d-e318458a5ac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:27:20.406533 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 514/1029] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:27:20 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 22:27:20.421012 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6549ab18-9a2f-409f-a3db-155d8740304f None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:20.421869 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6549ab18-9a2f-409f-a3db-155d8740304f None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:20.422135 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6549ab18-9a2f-409f-a3db-155d8740304f None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:20.423703 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 516/1030] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:27:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:27:20.428616 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ad7755-40c3-4098-a53f-6f62466012f4 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:20.428920 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ad7755-40c3-4098-a53f-6f62466012f4 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:20.429207 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ad7755-40c3-4098-a53f-6f62466012f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:20.433657 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6549ab18-9a2f-409f-a3db-155d8740304f None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:20.446919 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-06ad7755-40c3-4098-a53f-6f62466012f4 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:27:20.448069 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-06ad7755-40c3-4098-a53f-6f62466012f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:27:20.478694 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:20.518690 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 515/1031] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:27:20 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 22:27:20.543058 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6549ab18-9a2f-409f-a3db-155d8740304f None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:20.543229 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ad7755-40c3-4098-a53f-6f62466012f4 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:20.579898 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b320fe-fe38-434d-bded-ff589e0340a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:20.581480 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b320fe-fe38-434d-bded-ff589e0340a1 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:20.581772 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b320fe-fe38-434d-bded-ff589e0340a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:20.582054 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b320fe-fe38-434d-bded-ff589e0340a1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:20.583274 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b320fe-fe38-434d-bded-ff589e0340a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:20.584751 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b320fe-fe38-434d-bded-ff589e0340a1 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:20.586979 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 517/1032] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Jul 20 22:27:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 22:27:20.592423 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b320fe-fe38-434d-bded-ff589e0340a1 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:20.621354 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:20.656111 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ad7755-40c3-4098-a53f-6f62466012f4 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:20.693530 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa5eee2-1388-4cca-be3c-536cf15f604d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:20.694363 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa5eee2-1388-4cca-be3c-536cf15f604d None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:20.694653 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa5eee2-1388-4cca-be3c-536cf15f604d None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:20.694932 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa5eee2-1388-4cca-be3c-536cf15f604d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:20.698916 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b320fe-fe38-434d-bded-ff589e0340a1 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:20.706685 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa5eee2-1388-4cca-be3c-536cf15f604d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 22:27:20.715481 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-2aa5eee2-1388-4cca-be3c-536cf15f604d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 22:27:20.715481 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:27:20.715481 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:27:20.715481 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:27:20.715481 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:27:20.715481 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:27:20.715481 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:27:20.715481 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:27:20.715481 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:27:20.715481 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:27:20.715481 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:27:20.715481 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:27:20.715481 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 22:27:20.715481 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 22:27:20.715481 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 22:27:20.715481 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 22:27:20.716422 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 22:27:20.716422 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 22:27:20.716422 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 22:27:20.716422 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 22:27:20.716422 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:27:20.716422 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:27:20.716422 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:27:20.716422 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:27:20.716422 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:27:20.716422 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:27:20.716422 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:27:20.716422 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:27:20.716422 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:27:20.716422 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:27:20.716422 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:27:20.716422 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:27:20.717275 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:27:20.717275 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:27:20.717275 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:27:20.717275 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:27:20.717275 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 22:27:20.717275 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 22:27:20.717275 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 22:27:20.717275 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 22:27:20.717275 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 22:27:20.717275 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 22:27:20.717275 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 22:27:20.717275 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:27:20.717275 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 516/1033] 10.222.0.81 () {56 vars in 1120 bytes} [Wed Jul 20 22:27:20 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 22:27:20.722618 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa5eee2-1388-4cca-be3c-536cf15f604d None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:20.738316 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72105ea2-a46e-4cfb-b000-b9d75d068b04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:20.738992 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72105ea2-a46e-4cfb-b000-b9d75d068b04 None admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:20.739315 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72105ea2-a46e-4cfb-b000-b9d75d068b04 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:20.739599 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72105ea2-a46e-4cfb-b000-b9d75d068b04 None admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:20.740315 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72105ea2-a46e-4cfb-b000-b9d75d068b04 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:20.741748 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72105ea2-a46e-4cfb-b000-b9d75d068b04 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:20.754389 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:20.827922 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa5eee2-1388-4cca-be3c-536cf15f604d None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:20.846453 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 518/1034] 10.222.0.81 () {60 vars in 1132 bytes} [Wed Jul 20 22:27:20 2022] POST /identity/v3/users => generated 285 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 22:27:20.864003 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641d31c4-4abe-45d4-ad5d-a2fdb8d1fab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:20.864721 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641d31c4-4abe-45d4-ad5d-a2fdb8d1fab0 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:20.865013 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641d31c4-4abe-45d4-ad5d-a2fdb8d1fab0 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:20.865296 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641d31c4-4abe-45d4-ad5d-a2fdb8d1fab0 None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:20.866156 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641d31c4-4abe-45d4-ad5d-a2fdb8d1fab0 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:27:20.866491 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641d31c4-4abe-45d4-ad5d-a2fdb8d1fab0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:20.868797 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641d31c4-4abe-45d4-ad5d-a2fdb8d1fab0 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:20.885397 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 517/1035] 10.222.0.81 () {56 vars in 1115 bytes} [Wed Jul 20 22:27:20 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 22:27:20.886832 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72105ea2-a46e-4cfb-b000-b9d75d068b04 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:20.979713 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:21.000774 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72105ea2-a46e-4cfb-b000-b9d75d068b04 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:21.038615 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94584e5e-5c89-4609-94d6-3066111d459e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:21.039487 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94584e5e-5c89-4609-94d6-3066111d459e None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:21.039773 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94584e5e-5c89-4609-94d6-3066111d459e None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:21.040062 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94584e5e-5c89-4609-94d6-3066111d459e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:21.051886 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94584e5e-5c89-4609-94d6-3066111d459e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:21.053328 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94584e5e-5c89-4609-94d6-3066111d459e None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:21.072025 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-94584e5e-5c89-4609-94d6-3066111d459e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:27:21.072025 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:27:21.072025 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:27:21.072025 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:27:21.072025 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:27:21.072025 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:27:21.072025 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:27:21.072025 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:27:21.072025 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:27:21.072025 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:27:21.072025 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:27:21.072025 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:27:21.072025 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 22:27:21.072025 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 22:27:21.072025 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 22:27:21.072955 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 22:27:21.072955 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:27:21.072955 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:27:21.072955 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:27:21.072955 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:27:21.072955 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:27:21.072955 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:27:21.072955 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:27:21.072955 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:27:21.072955 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:27:21.072955 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:27:21.072955 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:27:21.072955 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:27:21.072955 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:27:21.072955 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:27:21.072955 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:27:21.072955 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:27:21.073845 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 22:27:21.073845 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 22:27:21.073845 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:27:21.073845 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:27:21.073845 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 519/1036] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Jul 20 22:27:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:27:21.079411 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641d31c4-4abe-45d4-ad5d-a2fdb8d1fab0 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:21.173621 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:21.191398 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641d31c4-4abe-45d4-ad5d-a2fdb8d1fab0 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:21.225226 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb4bf3e-5c8f-41f1-815c-2caedfdbaba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:21.225903 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb4bf3e-5c8f-41f1-815c-2caedfdbaba0 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:21.226188 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb4bf3e-5c8f-41f1-815c-2caedfdbaba0 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:21.226467 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb4bf3e-5c8f-41f1-815c-2caedfdbaba0 None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:21.227521 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb4bf3e-5c8f-41f1-815c-2caedfdbaba0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:27:21.228025 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb4bf3e-5c8f-41f1-815c-2caedfdbaba0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:21.229821 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb4bf3e-5c8f-41f1-815c-2caedfdbaba0 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:21.246255 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 518/1037] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Jul 20 22:27:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:27:21.247577 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94584e5e-5c89-4609-94d6-3066111d459e None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:21.352426 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94584e5e-5c89-4609-94d6-3066111d459e None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:21.369098 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba1e596-48e9-4522-adec-637c441772eb None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:21.369586 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba1e596-48e9-4522-adec-637c441772eb None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:21.369689 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba1e596-48e9-4522-adec-637c441772eb None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:21.371159 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 519/1038] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:27:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 22:27:21.376017 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96763715-fea8-4ecf-aa39-0d6e39445804 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:21.376307 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96763715-fea8-4ecf-aa39-0d6e39445804 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:21.376593 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96763715-fea8-4ecf-aa39-0d6e39445804 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:21.394169 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442fbf32-dab2-4cfb-b2d0-f48e68ad709a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:21.394560 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-96763715-fea8-4ecf-aa39-0d6e39445804 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:27:21.395448 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442fbf32-dab2-4cfb-b2d0-f48e68ad709a None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:21.395795 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-96763715-fea8-4ecf-aa39-0d6e39445804 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:27:21.396045 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442fbf32-dab2-4cfb-b2d0-f48e68ad709a None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:21.396470 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442fbf32-dab2-4cfb-b2d0-f48e68ad709a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:21.400096 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:21.407888 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442fbf32-dab2-4cfb-b2d0-f48e68ad709a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:21.412966 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442fbf32-dab2-4cfb-b2d0-f48e68ad709a None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:21.430429 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-442fbf32-dab2-4cfb-b2d0-f48e68ad709a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 20 22:27:21.430429 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:27:21.430429 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:27:21.430429 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:27:21.430429 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:27:21.430429 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:27:21.430429 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:27:21.430429 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:27:21.430429 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:27:21.430429 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:27:21.430429 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:27:21.430429 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:27:21.430429 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 22:27:21.430429 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 22:27:21.430429 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 22:27:21.430429 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 22:27:21.431732 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:27:21.431732 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:27:21.431732 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 22:27:21.431732 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 22:27:21.431732 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 22:27:21.431732 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 22:27:21.431732 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:27:21.431732 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:27:21.431732 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:27:21.431732 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:27:21.431732 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:27:21.431732 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:27:21.431732 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:27:21.431732 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:27:21.431732 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:27:21.431732 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:27:21.431732 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:27:21.431732 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:27:21.432911 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:27:21.432911 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:27:21.432911 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 22:27:21.432911 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 22:27:21.432911 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 22:27:21.432911 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 22:27:21.432911 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 22:27:21.432911 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 22:27:21.432911 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 20 22:27:21.432911 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:27:21.435886 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 520/1039] 10.222.0.81 () {56 vars in 1093 bytes} [Wed Jul 20 22:27:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:27:21.441919 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442fbf32-dab2-4cfb-b2d0-f48e68ad709a None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:21.471057 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 520/1040] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:27:21 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 22:27:21.492001 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96763715-fea8-4ecf-aa39-0d6e39445804 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:21.558108 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442fbf32-dab2-4cfb-b2d0-f48e68ad709a None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:21.574529 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:21.598142 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e3baa8-ebea-4768-a863-4314002beee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:21.598484 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96763715-fea8-4ecf-aa39-0d6e39445804 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:21.598817 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e3baa8-ebea-4768-a863-4314002beee0 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:21.599126 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e3baa8-ebea-4768-a863-4314002beee0 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:21.599405 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e3baa8-ebea-4768-a863-4314002beee0 None admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:21.600228 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e3baa8-ebea-4768-a863-4314002beee0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:27:21.601223 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e3baa8-ebea-4768-a863-4314002beee0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:21.602168 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e3baa8-ebea-4768-a863-4314002beee0 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:21.620064 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 521/1041] 10.222.0.81 () {56 vars in 1133 bytes} [Wed Jul 20 22:27:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:27:21.625659 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e3baa8-ebea-4768-a863-4314002beee0 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:21.634135 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b809922-c95f-454c-a7ee-e57ac6c8d8fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:21.634957 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b809922-c95f-454c-a7ee-e57ac6c8d8fb None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:21.635288 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b809922-c95f-454c-a7ee-e57ac6c8d8fb None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:21.635566 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b809922-c95f-454c-a7ee-e57ac6c8d8fb None admin] PATH_INFO: `/v3/services/image` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:21.636218 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b809922-c95f-454c-a7ee-e57ac6c8d8fb None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:21.637590 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b809922-c95f-454c-a7ee-e57ac6c8d8fb None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:21.650754 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-6b809922-c95f-454c-a7ee-e57ac6c8d8fb None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 20 22:27:21.650754 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:27:21.650754 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 22:27:21.650754 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 22:27:21.650754 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 22:27:21.650754 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 22:27:21.650754 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 22:27:21.650754 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 22:27:21.650754 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 20 22:27:21.650754 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:27:21.650754 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 22:27:21.650754 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:27:21.650754 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:27:21.650754 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:27:21.650754 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:27:21.650754 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:27:21.650754 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:27:21.651952 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:27:21.651952 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:27:21.651952 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:27:21.651952 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:27:21.651952 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:27:21.651952 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:27:21.651952 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 22:27:21.651952 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 22:27:21.651952 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 22:27:21.651952 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 22:27:21.651952 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:27:21.651952 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:27:21.651952 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:27:21.651952 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:27:21.651952 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:27:21.651952 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:27:21.656760 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:27:21.656760 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:27:21.656760 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:27:21.656760 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:27:21.656760 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:27:21.656760 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:27:21.656760 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:27:21.656760 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:27:21.656760 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:27:21.656760 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:27:21.656760 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 22:27:21.656760 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 22:27:21.656760 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 20 22:27:21.656760 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:27:21.656760 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 521/1042] 10.222.0.81 () {56 vars in 1105 bytes} [Wed Jul 20 22:27:21 2022] GET /identity/v3/services/image => generated 86 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 22:27:21.658312 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b809922-c95f-454c-a7ee-e57ac6c8d8fb None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:21.688651 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:21.741709 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e3baa8-ebea-4768-a863-4314002beee0 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:21.761708 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b809922-c95f-454c-a7ee-e57ac6c8d8fb None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:21.777316 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb84475-11bb-432f-8761-84c840313145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:21.778108 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb84475-11bb-432f-8761-84c840313145 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:21.778400 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb84475-11bb-432f-8761-84c840313145 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:21.778679 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb84475-11bb-432f-8761-84c840313145 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:21.798136 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363683b3-fc8e-41f1-a9ff-b9e59372364e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:21.798730 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363683b3-fc8e-41f1-a9ff-b9e59372364e None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:21.799034 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363683b3-fc8e-41f1-a9ff-b9e59372364e None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:21.799469 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363683b3-fc8e-41f1-a9ff-b9e59372364e None admin] PATH_INFO: `/v3/services` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:21.800023 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363683b3-fc8e-41f1-a9ff-b9e59372364e None admin] RBAC: Adding query filter params (name=image) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:27:21.800336 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363683b3-fc8e-41f1-a9ff-b9e59372364e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:21.801701 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363683b3-fc8e-41f1-a9ff-b9e59372364e None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:21.811307 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 522/1043] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Jul 20 22:27:21 2022] GET /identity/v3/services?name=image => generated 122 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 22:27:21.813342 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363683b3-fc8e-41f1-a9ff-b9e59372364e None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:21.822491 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:21.860465 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb84475-11bb-432f-8761-84c840313145 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:21.862497 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb84475-11bb-432f-8761-84c840313145 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:21.916811 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363683b3-fc8e-41f1-a9ff-b9e59372364e None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:21.917446 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:21.953736 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94438db7-d2a4-4907-9b57-b2f08f91fa66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:21.954408 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94438db7-d2a4-4907-9b57-b2f08f91fa66 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:21.954695 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94438db7-d2a4-4907-9b57-b2f08f91fa66 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:21.954968 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94438db7-d2a4-4907-9b57-b2f08f91fa66 None admin] PATH_INFO: `/v3/services` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:21.955693 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94438db7-d2a4-4907-9b57-b2f08f91fa66 None admin] RBAC: Adding query filter params (type=image) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:27:21.955997 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94438db7-d2a4-4907-9b57-b2f08f91fa66 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:21.957355 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94438db7-d2a4-4907-9b57-b2f08f91fa66 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:21.967110 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 523/1044] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Jul 20 22:27:21 2022] GET /identity/v3/services?type=image => generated 350 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 22:27:21.968046 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94438db7-d2a4-4907-9b57-b2f08f91fa66 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:21.973451 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 522/1045] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Jul 20 22:27:21 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 20 22:27:21.978748 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb84475-11bb-432f-8761-84c840313145 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:22.043349 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:22.075394 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94438db7-d2a4-4907-9b57-b2f08f91fa66 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:22.093647 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb84475-11bb-432f-8761-84c840313145 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:22.111690 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f0fdd2-fc33-4e4a-b476-9a36c73fb09b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:22.112352 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f0fdd2-fc33-4e4a-b476-9a36c73fb09b None admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:22.112636 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f0fdd2-fc33-4e4a-b476-9a36c73fb09b None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:22.112912 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f0fdd2-fc33-4e4a-b476-9a36c73fb09b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:22.113636 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f0fdd2-fc33-4e4a-b476-9a36c73fb09b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:22.115072 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f0fdd2-fc33-4e4a-b476-9a36c73fb09b None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:22.129594 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16850c1-dddd-4f41-80fa-b20466894a53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:22.130406 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16850c1-dddd-4f41-80fa-b20466894a53 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:22.130694 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16850c1-dddd-4f41-80fa-b20466894a53 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:22.130975 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16850c1-dddd-4f41-80fa-b20466894a53 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:22.141374 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16850c1-dddd-4f41-80fa-b20466894a53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:22.142810 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16850c1-dddd-4f41-80fa-b20466894a53 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:22.162753 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-a16850c1-dddd-4f41-80fa-b20466894a53 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:27:22.162753 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:27:22.162753 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:27:22.162753 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:27:22.162753 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:27:22.162753 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:27:22.162753 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:27:22.162753 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:27:22.162753 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:27:22.162753 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:27:22.162753 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:27:22.162753 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:27:22.162753 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 22:27:22.162753 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 22:27:22.162753 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 22:27:22.163655 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 22:27:22.163655 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:27:22.163655 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:27:22.163655 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:27:22.163655 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:27:22.163655 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:27:22.163655 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:27:22.163655 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:27:22.163655 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:27:22.163655 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:27:22.163655 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:27:22.163655 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:27:22.163655 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:27:22.163655 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:27:22.163655 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:27:22.163655 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:27:22.163655 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:27:22.164879 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 22:27:22.164879 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 22:27:22.164879 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:27:22.164879 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:27:22.164879 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 523/1046] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Jul 20 22:27:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 22:27:22.169539 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16850c1-dddd-4f41-80fa-b20466894a53 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:22.173550 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:22.222102 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 524/1047] 10.222.0.81 () {60 vars in 1144 bytes} [Wed Jul 20 22:27:21 2022] POST /identity/v3/endpoints => generated 327 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 22:27:22.285358 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16850c1-dddd-4f41-80fa-b20466894a53 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:22.328170 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c4d562-1cea-4ff9-8819-f92d3cb97346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:22.328853 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c4d562-1cea-4ff9-8819-f92d3cb97346 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:22.329136 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c4d562-1cea-4ff9-8819-f92d3cb97346 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:22.329410 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c4d562-1cea-4ff9-8819-f92d3cb97346 None admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:22.330137 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c4d562-1cea-4ff9-8819-f92d3cb97346 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:27:22.330449 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c4d562-1cea-4ff9-8819-f92d3cb97346 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:22.331867 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c4d562-1cea-4ff9-8819-f92d3cb97346 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:22.345028 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 524/1048] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Jul 20 22:27:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 22:27:22.350581 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f0fdd2-fc33-4e4a-b476-9a36c73fb09b None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:22.458456 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f0fdd2-fc33-4e4a-b476-9a36c73fb09b None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:22.484753 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:22.493500 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db126c08-6769-4cd0-8a12-8638a4779394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:22.499843 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db126c08-6769-4cd0-8a12-8638a4779394 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:22.499843 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db126c08-6769-4cd0-8a12-8638a4779394 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:22.499843 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db126c08-6769-4cd0-8a12-8638a4779394 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:22.502279 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db126c08-6769-4cd0-8a12-8638a4779394 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:22.503881 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db126c08-6769-4cd0-8a12-8638a4779394 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:22.518953 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-db126c08-6769-4cd0-8a12-8638a4779394 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 22:27:22.518953 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:27:22.518953 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:27:22.518953 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:27:22.518953 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:27:22.518953 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:27:22.518953 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:27:22.518953 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:27:22.518953 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:27:22.518953 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:27:22.518953 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:27:22.518953 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:27:22.518953 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 22:27:22.518953 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 22:27:22.518953 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 22:27:22.520089 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 22:27:22.520089 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:27:22.520089 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:27:22.520089 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:27:22.520089 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:27:22.520089 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:27:22.520089 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:27:22.520089 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:27:22.520089 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:27:22.520089 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:27:22.520089 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:27:22.520089 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:27:22.520089 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:27:22.520089 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:27:22.520089 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:27:22.520089 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:27:22.520089 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:27:22.521087 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 22:27:22.521087 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 22:27:22.521087 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 22:27:22.521087 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 22:27:22.521087 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 22:27:22.521087 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 22:27:22.521087 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 22:27:22.521087 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:27:22.523883 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 525/1049] 10.222.0.81 () {56 vars in 1111 bytes} [Wed Jul 20 22:27:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:27:22.529144 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c4d562-1cea-4ff9-8819-f92d3cb97346 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:22.593240 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:22.635410 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c4d562-1cea-4ff9-8819-f92d3cb97346 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:22.669475 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33367665-5c86-4022-820a-8ce25ef24fc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:22.670143 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33367665-5c86-4022-820a-8ce25ef24fc6 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:22.670429 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33367665-5c86-4022-820a-8ce25ef24fc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:22.670711 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33367665-5c86-4022-820a-8ce25ef24fc6 None admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:22.672087 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33367665-5c86-4022-820a-8ce25ef24fc6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:27:22.672403 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33367665-5c86-4022-820a-8ce25ef24fc6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:22.673826 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33367665-5c86-4022-820a-8ce25ef24fc6 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:22.687194 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 525/1050] 10.222.0.81 () {56 vars in 1148 bytes} [Wed Jul 20 22:27:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 22:27:22.692807 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db126c08-6769-4cd0-8a12-8638a4779394 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:22.795202 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db126c08-6769-4cd0-8a12-8638a4779394 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:22.829959 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af7f99a-12bc-412f-bd00-f5875b48608b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:22.830630 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af7f99a-12bc-412f-bd00-f5875b48608b None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:22.830918 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af7f99a-12bc-412f-bd00-f5875b48608b None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:22.831225 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af7f99a-12bc-412f-bd00-f5875b48608b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:22.832060 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af7f99a-12bc-412f-bd00-f5875b48608b None admin] RBAC: Adding query filter params (role.id=a3e411d61a234343b961e8bf78a55a11, scope.project.id=385edf2e59e546d090ace36600e29032, user.id=7829324db2574d38b18c749d0559177f) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:27:22.832454 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af7f99a-12bc-412f-bd00-f5875b48608b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:22.833958 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af7f99a-12bc-412f-bd00-f5875b48608b None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:22.847372 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 526/1051] 10.222.0.81 () {56 vars in 1374 bytes} [Wed Jul 20 22:27:22 2022] GET /identity/v3/role_assignments?user.id=7829324db2574d38b18c749d0559177f&scope.project.id=385edf2e59e546d090ace36600e29032&role.id=a3e411d61a234343b961e8bf78a55a11 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 22:27:22.990532 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bc891d-5ee2-4150-ad84-68cc873165f0 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:22.990853 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bc891d-5ee2-4150-ad84-68cc873165f0 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:22.991126 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bc891d-5ee2-4150-ad84-68cc873165f0 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:22.992793 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 526/1052] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:27:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:27:22.998156 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ebe9aa-e640-414a-8138-eb7137f70cba None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:22.998452 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ebe9aa-e640-414a-8138-eb7137f70cba None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:22.998732 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ebe9aa-e640-414a-8138-eb7137f70cba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:23.016625 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-e1ebe9aa-e640-414a-8138-eb7137f70cba None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:27:23.017816 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-e1ebe9aa-e640-414a-8138-eb7137f70cba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:27:23.064439 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:23.085222 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 527/1053] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:27:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 22:27:23.109643 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bc891d-5ee2-4150-ad84-68cc873165f0 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:23.221128 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bc891d-5ee2-4150-ad84-68cc873165f0 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:23.241807 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:23.256490 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e4f1a5-0ec8-43f6-a140-0153d689e16c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:23.257302 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e4f1a5-0ec8-43f6-a140-0153d689e16c None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:23.257586 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e4f1a5-0ec8-43f6-a140-0153d689e16c None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:23.257866 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e4f1a5-0ec8-43f6-a140-0153d689e16c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:23.267730 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e4f1a5-0ec8-43f6-a140-0153d689e16c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 22:27:23.275755 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-c7e4f1a5-0ec8-43f6-a140-0153d689e16c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 22:27:23.275755 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:27:23.275755 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:27:23.275755 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:27:23.275755 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:27:23.275755 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:27:23.275755 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:27:23.275755 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:27:23.275755 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:27:23.275755 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:27:23.275755 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:27:23.275755 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:27:23.275755 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 22:27:23.275755 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 22:27:23.275755 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 22:27:23.275755 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 22:27:23.276668 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 22:27:23.276668 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 22:27:23.276668 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 22:27:23.276668 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 22:27:23.276668 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:27:23.276668 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:27:23.276668 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:27:23.276668 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:27:23.276668 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:27:23.276668 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:27:23.276668 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:27:23.276668 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:27:23.276668 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:27:23.276668 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:27:23.276668 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:27:23.276668 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:27:23.277882 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:27:23.277882 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:27:23.277882 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:27:23.277882 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:27:23.277882 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 22:27:23.277882 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 22:27:23.277882 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 22:27:23.277882 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 22:27:23.277882 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 22:27:23.277882 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 22:27:23.277882 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 22:27:23.277882 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:27:23.279354 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 527/1054] 10.222.0.81 () {56 vars in 1096 bytes} [Wed Jul 20 22:27:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:27:23.283188 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ebe9aa-e640-414a-8138-eb7137f70cba None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:23.392462 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ebe9aa-e640-414a-8138-eb7137f70cba None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:23.429757 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b605d39e-24ce-4039-8ad6-aac838de100c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:23.430435 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b605d39e-24ce-4039-8ad6-aac838de100c None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:23.430723 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b605d39e-24ce-4039-8ad6-aac838de100c None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:23.431019 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b605d39e-24ce-4039-8ad6-aac838de100c None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:23.431766 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b605d39e-24ce-4039-8ad6-aac838de100c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:27:23.432077 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b605d39e-24ce-4039-8ad6-aac838de100c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:23.433440 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b605d39e-24ce-4039-8ad6-aac838de100c None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:23.446573 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 528/1055] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Jul 20 22:27:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:27:23.447818 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e4f1a5-0ec8-43f6-a140-0153d689e16c None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:23.550023 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e4f1a5-0ec8-43f6-a140-0153d689e16c None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:23.583523 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33089879-37a2-4c46-b987-9b407bc039f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:23.584403 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33089879-37a2-4c46-b987-9b407bc039f5 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:23.584687 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33089879-37a2-4c46-b987-9b407bc039f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:23.584965 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33089879-37a2-4c46-b987-9b407bc039f5 None admin] PATH_INFO: `/v3/users/169f6a90d744404ab6970d012e804eb5` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:23.599928 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33089879-37a2-4c46-b987-9b407bc039f5 None admin] RBAC: Authorizing `identity:get_user(user_id=169f6a90d744404ab6970d012e804eb5)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:23.601365 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33089879-37a2-4c46-b987-9b407bc039f5 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:23.607907 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 528/1056] 10.222.0.81 () {56 vars in 1177 bytes} [Wed Jul 20 22:27:23 2022] GET /identity/v3/users/169f6a90d744404ab6970d012e804eb5 => generated 285 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:27:23.609585 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b605d39e-24ce-4039-8ad6-aac838de100c None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:23.652301 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:23.708777 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b605d39e-24ce-4039-8ad6-aac838de100c None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:23.742402 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aa3dc5-a674-4b59-a61c-f84cc45b67ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:23.743091 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aa3dc5-a674-4b59-a61c-f84cc45b67ac None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:23.743383 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aa3dc5-a674-4b59-a61c-f84cc45b67ac None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:23.743664 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aa3dc5-a674-4b59-a61c-f84cc45b67ac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:23.810595 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79aa3dc5-a674-4b59-a61c-f84cc45b67ac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:23.812121 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79aa3dc5-a674-4b59-a61c-f84cc45b67ac None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:23.853966 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:23.913810 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 529/1057] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Jul 20 22:27:23 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 22:27:23.919597 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33089879-37a2-4c46-b987-9b407bc039f5 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:23.963851 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:24.023996 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33089879-37a2-4c46-b987-9b407bc039f5 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:24.058293 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7989a05f-5385-4c5e-a4db-23db991a01bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:24.059210 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7989a05f-5385-4c5e-a4db-23db991a01bf None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:24.059543 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7989a05f-5385-4c5e-a4db-23db991a01bf None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:24.059818 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7989a05f-5385-4c5e-a4db-23db991a01bf None admin] PATH_INFO: `/v3/projects/1db282486e4e45188be54680737fd949` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:24.060988 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7989a05f-5385-4c5e-a4db-23db991a01bf None admin] RBAC: Authorizing `identity:get_project(project_id=1db282486e4e45188be54680737fd949)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:24.063066 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7989a05f-5385-4c5e-a4db-23db991a01bf None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:24.069775 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 529/1058] 10.222.0.81 () {56 vars in 1186 bytes} [Wed Jul 20 22:27:23 2022] GET /identity/v3/projects/1db282486e4e45188be54680737fd949 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 22:27:24.071060 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aa3dc5-a674-4b59-a61c-f84cc45b67ac None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:24.168502 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aa3dc5-a674-4b59-a61c-f84cc45b67ac None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:24.211122 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1b07f8-b742-42cd-ae88-c200403616f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:24.211881 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1b07f8-b742-42cd-ae88-c200403616f8 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:24.212176 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1b07f8-b742-42cd-ae88-c200403616f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:24.212453 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1b07f8-b742-42cd-ae88-c200403616f8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:24.213276 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1b07f8-b742-42cd-ae88-c200403616f8 None admin] RBAC: Adding query filter params (role.id=d89b6c92d35a401a8517f1d47f6be843, scope.project.id=1db282486e4e45188be54680737fd949, user.id=169f6a90d744404ab6970d012e804eb5) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:27:24.213588 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1b07f8-b742-42cd-ae88-c200403616f8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:24.215041 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1b07f8-b742-42cd-ae88-c200403616f8 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:24.221348 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 530/1059] 10.222.0.81 () {56 vars in 1374 bytes} [Wed Jul 20 22:27:24 2022] GET /identity/v3/role_assignments?user.id=169f6a90d744404ab6970d012e804eb5&scope.project.id=1db282486e4e45188be54680737fd949&role.id=d89b6c92d35a401a8517f1d47f6be843 => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:27:24.322118 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b98055-217b-4004-9613-acf57d51e32d None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:24.322419 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b98055-217b-4004-9613-acf57d51e32d None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:24.322845 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b98055-217b-4004-9613-acf57d51e32d None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:24.324359 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 530/1060] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:27:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:27:24.330603 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a98040b-d1d0-47ae-86f8-daeedc47e294 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:24.330901 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a98040b-d1d0-47ae-86f8-daeedc47e294 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:24.331218 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a98040b-d1d0-47ae-86f8-daeedc47e294 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:24.343645 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:24.350075 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-9a98040b-d1d0-47ae-86f8-daeedc47e294 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:27:24.351338 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-9a98040b-d1d0-47ae-86f8-daeedc47e294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:27:24.421701 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 531/1061] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:27:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 22:27:24.444571 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b98055-217b-4004-9613-acf57d51e32d None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:24.549136 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b98055-217b-4004-9613-acf57d51e32d None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:24.556851 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:24.584956 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bb9041-49b8-4f2a-ac4a-5321f5759fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:24.585782 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bb9041-49b8-4f2a-ac4a-5321f5759fae None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:24.586100 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bb9041-49b8-4f2a-ac4a-5321f5759fae None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:24.586627 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bb9041-49b8-4f2a-ac4a-5321f5759fae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:24.656307 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bb9041-49b8-4f2a-ac4a-5321f5759fae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:24.657777 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bb9041-49b8-4f2a-ac4a-5321f5759fae None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:24.760227 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 531/1062] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Jul 20 22:27:24 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 22:27:24.766366 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a98040b-d1d0-47ae-86f8-daeedc47e294 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:24.868470 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a98040b-d1d0-47ae-86f8-daeedc47e294 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:24.903655 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1ddf66-9551-40cc-ab1e-112dab18f8da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:24.904470 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1ddf66-9551-40cc-ab1e-112dab18f8da None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:24.904758 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1ddf66-9551-40cc-ab1e-112dab18f8da None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:24.905554 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1ddf66-9551-40cc-ab1e-112dab18f8da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:24.908282 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:24.916895 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1ddf66-9551-40cc-ab1e-112dab18f8da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:24.918356 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1ddf66-9551-40cc-ab1e-112dab18f8da None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:24.923253 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f802549-76d2-4fc7-9d13-d787f03ce9ba None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:24.923253 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f802549-76d2-4fc7-9d13-d787f03ce9ba None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:24.923253 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f802549-76d2-4fc7-9d13-d787f03ce9ba None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:24.923253 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 532/1063] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:27:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:27:24.926987 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9b980d-0251-438a-b921-a108f723744b None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:24.927304 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9b980d-0251-438a-b921-a108f723744b None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:24.927593 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9b980d-0251-438a-b921-a108f723744b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:24.937486 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-3f1ddf66-9551-40cc-ab1e-112dab18f8da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:27:24.937486 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:27:24.937486 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:27:24.937486 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:27:24.937486 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:27:24.937486 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:27:24.937486 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:27:24.937486 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:27:24.937486 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:27:24.937486 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:27:24.937486 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:27:24.937486 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:27:24.937486 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 22:27:24.937486 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 22:27:24.937486 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 22:27:24.938455 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 22:27:24.938455 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:27:24.938455 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:27:24.938455 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:27:24.938455 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:27:24.938455 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:27:24.938455 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:27:24.938455 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:27:24.938455 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:27:24.938455 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:27:24.938455 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:27:24.938455 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:27:24.938455 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:27:24.938455 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:27:24.938455 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:27:24.938455 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:27:24.938455 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:27:24.939373 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 22:27:24.939373 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 22:27:24.939373 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:27:24.939373 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:27:24.939373 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 532/1064] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Jul 20 22:27:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 22:27:24.939658 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:24.944692 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1ddf66-9551-40cc-ab1e-112dab18f8da None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:24.945244 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-fc9b980d-0251-438a-b921-a108f723744b None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:27:24.949447 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-fc9b980d-0251-438a-b921-a108f723744b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:27:25.023422 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 533/1065] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:27:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 22:27:25.056586 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9b980d-0251-438a-b921-a108f723744b None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:25.066903 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1ddf66-9551-40cc-ab1e-112dab18f8da None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:25.110489 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c237be3-4e9a-46ec-aefd-c25363197c79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:25.111215 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c237be3-4e9a-46ec-aefd-c25363197c79 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:25.111484 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c237be3-4e9a-46ec-aefd-c25363197c79 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:25.111766 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c237be3-4e9a-46ec-aefd-c25363197c79 None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:25.112484 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c237be3-4e9a-46ec-aefd-c25363197c79 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:27:25.112796 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c237be3-4e9a-46ec-aefd-c25363197c79 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:25.114161 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c237be3-4e9a-46ec-aefd-c25363197c79 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:25.126337 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 533/1066] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Jul 20 22:27:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 22:27:25.168013 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9b980d-0251-438a-b921-a108f723744b None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:25.212795 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6e1839-e8db-42b9-8b16-893fe3022a36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:25.213636 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6e1839-e8db-42b9-8b16-893fe3022a36 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:25.213922 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6e1839-e8db-42b9-8b16-893fe3022a36 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:25.214206 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6e1839-e8db-42b9-8b16-893fe3022a36 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:25.218267 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:25.226986 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6e1839-e8db-42b9-8b16-893fe3022a36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 22:27:25.236645 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-7b6e1839-e8db-42b9-8b16-893fe3022a36 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 22:27:25.236645 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:27:25.236645 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:27:25.236645 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:27:25.236645 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:27:25.236645 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:27:25.236645 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:27:25.236645 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:27:25.236645 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:27:25.236645 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:27:25.236645 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:27:25.236645 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:27:25.236645 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 22:27:25.236645 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 22:27:25.236645 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 22:27:25.236645 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 22:27:25.237690 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 22:27:25.237690 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 22:27:25.237690 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 22:27:25.237690 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 22:27:25.237690 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:27:25.237690 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:27:25.237690 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:27:25.237690 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:27:25.237690 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:27:25.237690 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:27:25.237690 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:27:25.237690 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:27:25.237690 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:27:25.237690 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:27:25.237690 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:27:25.237690 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:27:25.238533 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:27:25.238533 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:27:25.238533 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:27:25.238533 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:27:25.238533 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 22:27:25.238533 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 22:27:25.238533 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 22:27:25.238533 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 22:27:25.238533 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 22:27:25.238533 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 22:27:25.238533 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 22:27:25.238533 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:27:25.238533 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 534/1067] 10.222.0.81 () {56 vars in 1120 bytes} [Wed Jul 20 22:27:25 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 22:27:25.245667 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c237be3-4e9a-46ec-aefd-c25363197c79 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:25.300482 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:25.355789 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c237be3-4e9a-46ec-aefd-c25363197c79 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:25.396624 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9624a80f-6d6c-4f73-a570-b78544fcdc91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:25.397685 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9624a80f-6d6c-4f73-a570-b78544fcdc91 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:25.397976 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9624a80f-6d6c-4f73-a570-b78544fcdc91 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:25.398255 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9624a80f-6d6c-4f73-a570-b78544fcdc91 None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:25.399034 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9624a80f-6d6c-4f73-a570-b78544fcdc91 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:27:25.399350 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9624a80f-6d6c-4f73-a570-b78544fcdc91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:25.400852 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9624a80f-6d6c-4f73-a570-b78544fcdc91 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:25.411163 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 534/1068] 10.222.0.81 () {56 vars in 1115 bytes} [Wed Jul 20 22:27:25 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:27:25.416845 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6e1839-e8db-42b9-8b16-893fe3022a36 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:25.520481 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6e1839-e8db-42b9-8b16-893fe3022a36 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:25.553901 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5df9587-78cf-4f8c-b9e6-3d10b533d5ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:25.554648 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5df9587-78cf-4f8c-b9e6-3d10b533d5ce None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:25.554951 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5df9587-78cf-4f8c-b9e6-3d10b533d5ce None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:25.555262 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5df9587-78cf-4f8c-b9e6-3d10b533d5ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:25.565621 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5df9587-78cf-4f8c-b9e6-3d10b533d5ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:25.567087 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5df9587-78cf-4f8c-b9e6-3d10b533d5ce None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:25.584696 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-d5df9587-78cf-4f8c-b9e6-3d10b533d5ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:27:25.584696 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:27:25.584696 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:27:25.584696 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:27:25.584696 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:27:25.584696 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:27:25.584696 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:27:25.584696 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:27:25.584696 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:27:25.584696 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:27:25.584696 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:27:25.584696 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:27:25.584696 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 22:27:25.584696 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 22:27:25.584696 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 22:27:25.585675 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 22:27:25.585675 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:27:25.585675 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:27:25.585675 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:27:25.585675 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:27:25.585675 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:27:25.585675 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:27:25.585675 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:27:25.585675 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:27:25.585675 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:27:25.585675 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:27:25.585675 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:27:25.585675 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:27:25.585675 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:27:25.585675 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:27:25.585675 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:27:25.585675 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:27:25.586589 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 22:27:25.586589 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 22:27:25.586589 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:27:25.586589 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:27:25.586589 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 535/1069] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Jul 20 22:27:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:27:25.592029 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9624a80f-6d6c-4f73-a570-b78544fcdc91 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:25.690413 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9624a80f-6d6c-4f73-a570-b78544fcdc91 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:25.700305 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:25.723211 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc0668d-128f-40c0-92ec-e08dd47c683f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:25.723887 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc0668d-128f-40c0-92ec-e08dd47c683f None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:25.724177 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc0668d-128f-40c0-92ec-e08dd47c683f None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:25.724455 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc0668d-128f-40c0-92ec-e08dd47c683f None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:25.725173 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc0668d-128f-40c0-92ec-e08dd47c683f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:27:25.725482 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc0668d-128f-40c0-92ec-e08dd47c683f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:25.726853 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc0668d-128f-40c0-92ec-e08dd47c683f None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:25.739488 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 535/1070] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Jul 20 22:27:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:27:25.745650 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5df9587-78cf-4f8c-b9e6-3d10b533d5ce None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:25.811602 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:25.857395 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5df9587-78cf-4f8c-b9e6-3d10b533d5ce None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:25.891369 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580ccbe0-1c26-4d56-9fac-5a4fcd9fae8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:25.892180 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580ccbe0-1c26-4d56-9fac-5a4fcd9fae8d None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:25.892465 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580ccbe0-1c26-4d56-9fac-5a4fcd9fae8d None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:25.892748 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580ccbe0-1c26-4d56-9fac-5a4fcd9fae8d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:25.900217 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580ccbe0-1c26-4d56-9fac-5a4fcd9fae8d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:25.901660 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580ccbe0-1c26-4d56-9fac-5a4fcd9fae8d None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:25.916607 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-580ccbe0-1c26-4d56-9fac-5a4fcd9fae8d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 20 22:27:25.916607 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:27:25.916607 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:27:25.916607 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:27:25.916607 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:27:25.916607 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:27:25.916607 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:27:25.916607 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:27:25.916607 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:27:25.916607 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:27:25.916607 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:27:25.916607 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:27:25.916607 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 22:27:25.916607 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 22:27:25.916607 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 22:27:25.916607 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 22:27:25.917654 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:27:25.917654 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:27:25.917654 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 22:27:25.917654 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 22:27:25.917654 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 22:27:25.917654 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 22:27:25.917654 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:27:25.917654 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:27:25.917654 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:27:25.917654 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:27:25.917654 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:27:25.917654 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:27:25.917654 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:27:25.917654 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:27:25.917654 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:27:25.917654 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:27:25.917654 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:27:25.917654 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:27:25.919058 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:27:25.919058 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:27:25.919058 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 22:27:25.919058 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 22:27:25.919058 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 22:27:25.919058 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 22:27:25.919058 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 22:27:25.919058 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 22:27:25.919058 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 20 22:27:25.919058 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:27:25.919058 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 536/1071] 10.222.0.81 () {56 vars in 1093 bytes} [Wed Jul 20 22:27:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 22:27:25.924328 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc0668d-128f-40c0-92ec-e08dd47c683f None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:26.021031 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc0668d-128f-40c0-92ec-e08dd47c683f None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:26.054870 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c1ef73-8d23-4e39-b746-c0bb31002b82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:26.055578 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c1ef73-8d23-4e39-b746-c0bb31002b82 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:26.055867 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c1ef73-8d23-4e39-b746-c0bb31002b82 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:26.056142 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c1ef73-8d23-4e39-b746-c0bb31002b82 None admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:26.056976 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c1ef73-8d23-4e39-b746-c0bb31002b82 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:27:26.057289 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c1ef73-8d23-4e39-b746-c0bb31002b82 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:26.058713 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c1ef73-8d23-4e39-b746-c0bb31002b82 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:26.081220 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 536/1072] 10.222.0.81 () {56 vars in 1133 bytes} [Wed Jul 20 22:27:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 22:27:26.086785 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580ccbe0-1c26-4d56-9fac-5a4fcd9fae8d None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:26.200569 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580ccbe0-1c26-4d56-9fac-5a4fcd9fae8d None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:26.209998 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:26.234740 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84c1b3b-ef93-499a-9183-ca67d4603d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:26.235574 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84c1b3b-ef93-499a-9183-ca67d4603d0d None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:26.235862 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84c1b3b-ef93-499a-9183-ca67d4603d0d None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:26.236139 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84c1b3b-ef93-499a-9183-ca67d4603d0d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:26.247057 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84c1b3b-ef93-499a-9183-ca67d4603d0d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:26.248504 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84c1b3b-ef93-499a-9183-ca67d4603d0d None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:26.266221 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-c84c1b3b-ef93-499a-9183-ca67d4603d0d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:27:26.266221 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:27:26.266221 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:27:26.266221 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:27:26.266221 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:27:26.266221 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:27:26.266221 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:27:26.266221 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:27:26.266221 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:27:26.266221 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:27:26.266221 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:27:26.266221 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:27:26.266221 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 22:27:26.266221 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 22:27:26.266221 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 22:27:26.267158 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 22:27:26.267158 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:27:26.267158 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:27:26.267158 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:27:26.267158 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:27:26.267158 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:27:26.267158 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:27:26.267158 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:27:26.267158 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:27:26.267158 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:27:26.267158 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:27:26.267158 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:27:26.267158 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:27:26.267158 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:27:26.267158 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:27:26.267158 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:27:26.267158 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:27:26.268187 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 22:27:26.268187 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 22:27:26.268187 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:27:26.268187 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:27:26.268187 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 537/1073] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Jul 20 22:27:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:27:26.273180 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c1ef73-8d23-4e39-b746-c0bb31002b82 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:26.295167 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ae1b92-d5f2-40d6-a71c-4c043218611b None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:26.295987 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ae1b92-d5f2-40d6-a71c-4c043218611b None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:26.296280 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ae1b92-d5f2-40d6-a71c-4c043218611b None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:26.297812 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 538/1074] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:27:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 22:27:26.302354 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8db775-677b-4418-abe4-7eb26a0c144c None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:26.302647 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8db775-677b-4418-abe4-7eb26a0c144c None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:26.302928 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8db775-677b-4418-abe4-7eb26a0c144c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:26.303596 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:26.320600 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-ce8db775-677b-4418-abe4-7eb26a0c144c None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:27:26.321798 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-ce8db775-677b-4418-abe4-7eb26a0c144c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:27:26.375787 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c1ef73-8d23-4e39-b746-c0bb31002b82 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:26.387301 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 539/1075] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:27:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 22:27:26.410373 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01028dd4-d046-4652-a43e-c2471a9e94e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:26.410714 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8db775-677b-4418-abe4-7eb26a0c144c None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:26.411071 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01028dd4-d046-4652-a43e-c2471a9e94e6 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:26.411359 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01028dd4-d046-4652-a43e-c2471a9e94e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:26.411638 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01028dd4-d046-4652-a43e-c2471a9e94e6 None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:26.412358 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01028dd4-d046-4652-a43e-c2471a9e94e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:27:26.412668 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01028dd4-d046-4652-a43e-c2471a9e94e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:26.414036 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01028dd4-d046-4652-a43e-c2471a9e94e6 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:26.426007 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 537/1076] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Jul 20 22:27:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:27:26.431716 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01028dd4-d046-4652-a43e-c2471a9e94e6 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:26.501879 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:26.511202 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8db775-677b-4418-abe4-7eb26a0c144c None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:26.520235 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:26.532079 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01028dd4-d046-4652-a43e-c2471a9e94e6 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:26.546682 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b96775-1640-4692-8623-3336a62f2ffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:26.547534 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b96775-1640-4692-8623-3336a62f2ffd None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:26.547821 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b96775-1640-4692-8623-3336a62f2ffd None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:26.548099 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b96775-1640-4692-8623-3336a62f2ffd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:26.558196 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b96775-1640-4692-8623-3336a62f2ffd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 22:27:26.565976 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3842b25b-ae4f-4583-baa1-60592942baa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:26.566314 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-48b96775-1640-4692-8623-3336a62f2ffd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 22:27:26.566314 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:27:26.566314 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:27:26.566314 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:27:26.566314 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:27:26.566314 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:27:26.566314 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:27:26.566314 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:27:26.566314 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:27:26.566314 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:27:26.566314 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:27:26.566314 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:27:26.566314 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 22:27:26.566314 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 22:27:26.566314 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 22:27:26.566314 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 22:27:26.567715 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 22:27:26.567715 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 22:27:26.567715 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 22:27:26.567715 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 22:27:26.567715 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:27:26.567715 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:27:26.567715 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:27:26.567715 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:27:26.567715 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:27:26.567715 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:27:26.567715 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:27:26.567715 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:27:26.567715 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:27:26.567715 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:27:26.567715 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:27:26.567715 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:27:26.568560 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:27:26.568560 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:27:26.568560 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:27:26.568560 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:27:26.568560 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 22:27:26.568560 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 22:27:26.568560 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 22:27:26.568560 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 22:27:26.568560 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 22:27:26.568560 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 22:27:26.568560 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 22:27:26.568560 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:27:26.569222 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3842b25b-ae4f-4583-baa1-60592942baa3 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:26.569222 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3842b25b-ae4f-4583-baa1-60592942baa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:26.569222 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3842b25b-ae4f-4583-baa1-60592942baa3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:26.569426 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 540/1077] 10.222.0.81 () {56 vars in 1096 bytes} [Wed Jul 20 22:27:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 22:27:26.573587 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b96775-1640-4692-8623-3336a62f2ffd None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:26.576245 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3842b25b-ae4f-4583-baa1-60592942baa3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:26.577681 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3842b25b-ae4f-4583-baa1-60592942baa3 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:26.593020 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-3842b25b-ae4f-4583-baa1-60592942baa3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 22:27:26.593020 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:27:26.593020 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:27:26.593020 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:27:26.593020 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:27:26.593020 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:27:26.593020 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:27:26.593020 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:27:26.593020 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:27:26.593020 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:27:26.593020 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:27:26.593020 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:27:26.593020 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 22:27:26.593020 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 22:27:26.593020 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 22:27:26.593962 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 22:27:26.593962 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:27:26.593962 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:27:26.593962 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:27:26.593962 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:27:26.593962 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:27:26.593962 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:27:26.593962 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:27:26.593962 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:27:26.593962 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:27:26.593962 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:27:26.593962 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:27:26.593962 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:27:26.593962 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:27:26.593962 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:27:26.593962 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:27:26.593962 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:27:26.594863 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 22:27:26.594863 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 22:27:26.594863 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 22:27:26.594863 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 22:27:26.594863 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 22:27:26.594863 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 22:27:26.594863 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 22:27:26.594863 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:27:26.594863 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 538/1078] 10.222.0.81 () {56 vars in 1111 bytes} [Wed Jul 20 22:27:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 22:27:26.599950 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3842b25b-ae4f-4583-baa1-60592942baa3 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:26.673105 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b96775-1640-4692-8623-3336a62f2ffd None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:26.696089 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3842b25b-ae4f-4583-baa1-60592942baa3 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:26.706207 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3106dc1e-f794-4a32-b0e7-a6cb45503103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:26.706897 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3106dc1e-f794-4a32-b0e7-a6cb45503103 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:26.707210 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3106dc1e-f794-4a32-b0e7-a6cb45503103 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:26.707490 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3106dc1e-f794-4a32-b0e7-a6cb45503103 None admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:26.708228 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3106dc1e-f794-4a32-b0e7-a6cb45503103 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:27:26.708544 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3106dc1e-f794-4a32-b0e7-a6cb45503103 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:26.709902 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3106dc1e-f794-4a32-b0e7-a6cb45503103 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:26.718450 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 541/1079] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Jul 20 22:27:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 22:27:26.724480 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3106dc1e-f794-4a32-b0e7-a6cb45503103 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:26.729825 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2644b7-1d03-446a-a845-1b05fc43266e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:26.730498 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2644b7-1d03-446a-a845-1b05fc43266e None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:26.730788 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2644b7-1d03-446a-a845-1b05fc43266e None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:26.731106 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2644b7-1d03-446a-a845-1b05fc43266e None admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:26.731894 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2644b7-1d03-446a-a845-1b05fc43266e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:27:26.732211 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2644b7-1d03-446a-a845-1b05fc43266e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:26.734192 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2644b7-1d03-446a-a845-1b05fc43266e None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:26.739343 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:26.745770 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 539/1080] 10.222.0.81 () {56 vars in 1148 bytes} [Wed Jul 20 22:27:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 22:27:26.751680 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2644b7-1d03-446a-a845-1b05fc43266e None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:26.762061 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:26.825190 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3106dc1e-f794-4a32-b0e7-a6cb45503103 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:26.849606 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2644b7-1d03-446a-a845-1b05fc43266e None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:26.908594 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07a91e0-0a56-4037-a981-731b8837e4c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:26.909750 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07a91e0-0a56-4037-a981-731b8837e4c0 None admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:26.910029 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07a91e0-0a56-4037-a981-731b8837e4c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:26.910199 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e63965-9b59-4e64-89c0-048f8e2a9a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:26.910311 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07a91e0-0a56-4037-a981-731b8837e4c0 None admin] PATH_INFO: `/v3/projects/385edf2e59e546d090ace36600e29032/users/7829324db2574d38b18c749d0559177f/roles/a3e411d61a234343b961e8bf78a55a11` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:26.911114 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e63965-9b59-4e64-89c0-048f8e2a9a7f None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:26.911403 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e63965-9b59-4e64-89c0-048f8e2a9a7f None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:26.911684 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e63965-9b59-4e64-89c0-048f8e2a9a7f None admin] PATH_INFO: `/v3/users/169f6a90d744404ab6970d012e804eb5` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:26.912556 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07a91e0-0a56-4037-a981-731b8837e4c0 None admin] RBAC: Authorizing `identity:create_grant(project_id=385edf2e59e546d090ace36600e29032, user_id=7829324db2574d38b18c749d0559177f, role_id=a3e411d61a234343b961e8bf78a55a11)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:26.912866 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e63965-9b59-4e64-89c0-048f8e2a9a7f None admin] RBAC: Authorizing `identity:get_user(user_id=169f6a90d744404ab6970d012e804eb5)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:26.913972 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07a91e0-0a56-4037-a981-731b8837e4c0 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:26.914269 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e63965-9b59-4e64-89c0-048f8e2a9a7f None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:26.915954 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 542/1081] 10.222.0.81 () {56 vars in 1177 bytes} [Wed Jul 20 22:27:26 2022] GET /identity/v3/users/169f6a90d744404ab6970d012e804eb5 => generated 285 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 22:27:26.922257 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e63965-9b59-4e64-89c0-048f8e2a9a7f None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:26.975590 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 540/1082] 10.222.0.81 () {58 vars in 1439 bytes} [Wed Jul 20 22:27:26 2022] PUT /identity/v3/projects/385edf2e59e546d090ace36600e29032/users/7829324db2574d38b18c749d0559177f/roles/a3e411d61a234343b961e8bf78a55a11 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 22:27:27.002339 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:27.022629 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e63965-9b59-4e64-89c0-048f8e2a9a7f None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:27.059910 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1c9096-4cbb-4404-a7c8-170df3b31811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:27.060795 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1c9096-4cbb-4404-a7c8-170df3b31811 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:27.061080 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1c9096-4cbb-4404-a7c8-170df3b31811 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:27.061388 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1c9096-4cbb-4404-a7c8-170df3b31811 None admin] PATH_INFO: `/v3/projects/1db282486e4e45188be54680737fd949` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:27.062623 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1c9096-4cbb-4404-a7c8-170df3b31811 None admin] RBAC: Authorizing `identity:get_project(project_id=1db282486e4e45188be54680737fd949)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:27.064106 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1c9096-4cbb-4404-a7c8-170df3b31811 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:27.066202 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 543/1083] 10.222.0.81 () {56 vars in 1186 bytes} [Wed Jul 20 22:27:26 2022] GET /identity/v3/projects/1db282486e4e45188be54680737fd949 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 22:27:27.072352 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07a91e0-0a56-4037-a981-731b8837e4c0 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:27.150777 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:27.183649 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07a91e0-0a56-4037-a981-731b8837e4c0 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:27.218046 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafe1efc-70d9-4503-8c95-cbb6bcc3fb92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:27.218982 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafe1efc-70d9-4503-8c95-cbb6bcc3fb92 None admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:27.219316 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafe1efc-70d9-4503-8c95-cbb6bcc3fb92 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:27.219617 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafe1efc-70d9-4503-8c95-cbb6bcc3fb92 None admin] PATH_INFO: `/v3/projects/1db282486e4e45188be54680737fd949/users/169f6a90d744404ab6970d012e804eb5/roles/d89b6c92d35a401a8517f1d47f6be843` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:27.221652 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafe1efc-70d9-4503-8c95-cbb6bcc3fb92 None admin] RBAC: Authorizing `identity:create_grant(project_id=1db282486e4e45188be54680737fd949, user_id=169f6a90d744404ab6970d012e804eb5, role_id=d89b6c92d35a401a8517f1d47f6be843)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:27.223206 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafe1efc-70d9-4503-8c95-cbb6bcc3fb92 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:27.302365 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 541/1084] 10.222.0.81 () {58 vars in 1439 bytes} [Wed Jul 20 22:27:27 2022] PUT /identity/v3/projects/1db282486e4e45188be54680737fd949/users/169f6a90d744404ab6970d012e804eb5/roles/d89b6c92d35a401a8517f1d47f6be843 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:27:28.992202 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357b4752-20a3-48cb-af60-349fdc1463f5 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:28.992777 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357b4752-20a3-48cb-af60-349fdc1463f5 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:28.992777 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357b4752-20a3-48cb-af60-349fdc1463f5 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:28.994111 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 544/1085] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:27:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:27:28.999747 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a25241f-5bd9-407f-9477-9b8a78535fca None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:29.000190 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a25241f-5bd9-407f-9477-9b8a78535fca None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:29.000455 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a25241f-5bd9-407f-9477-9b8a78535fca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:29.017520 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-6a25241f-5bd9-407f-9477-9b8a78535fca None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:27:29.018714 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-6a25241f-5bd9-407f-9477-9b8a78535fca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:27:29.083147 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 542/1086] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:27:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 22:27:29.106957 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357b4752-20a3-48cb-af60-349fdc1463f5 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:29.210788 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357b4752-20a3-48cb-af60-349fdc1463f5 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:29.244093 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f6f48b-2930-41ee-95cc-7116aa814691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:29.244846 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f6f48b-2930-41ee-95cc-7116aa814691 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:29.245131 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f6f48b-2930-41ee-95cc-7116aa814691 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:29.245408 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f6f48b-2930-41ee-95cc-7116aa814691 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:29.255820 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f6f48b-2930-41ee-95cc-7116aa814691 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 22:27:29.264436 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-e5f6f48b-2930-41ee-95cc-7116aa814691 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 22:27:29.264436 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:27:29.264436 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:27:29.264436 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:27:29.264436 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:27:29.264436 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:27:29.264436 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:27:29.264436 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:27:29.264436 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:27:29.264436 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:27:29.264436 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:27:29.264436 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:27:29.264436 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 22:27:29.264436 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 22:27:29.264436 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 22:27:29.264436 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 22:27:29.265599 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 22:27:29.265599 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 22:27:29.265599 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 22:27:29.265599 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 22:27:29.265599 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:27:29.265599 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:27:29.265599 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:27:29.265599 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:27:29.265599 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:27:29.265599 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:27:29.265599 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:27:29.265599 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:27:29.265599 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:27:29.265599 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:27:29.265599 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:27:29.265599 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:27:29.266521 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:27:29.266521 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:27:29.266521 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:27:29.266521 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:27:29.266521 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 22:27:29.266521 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 22:27:29.266521 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 22:27:29.266521 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 22:27:29.266521 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 22:27:29.266521 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 22:27:29.266521 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 22:27:29.266521 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:27:29.266521 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 545/1087] 10.222.0.81 () {56 vars in 1120 bytes} [Wed Jul 20 22:27:29 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:27:29.271556 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a25241f-5bd9-407f-9477-9b8a78535fca None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:29.320336 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1079a377-6330-480e-b4b8-18b75d23c214 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:29.320623 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1079a377-6330-480e-b4b8-18b75d23c214 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:29.320900 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1079a377-6330-480e-b4b8-18b75d23c214 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:29.322270 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 546/1088] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:27:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 22:27:29.327733 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0c6afc-fbc9-4f6a-ad3a-d8966e66c3d4 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:29.328025 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0c6afc-fbc9-4f6a-ad3a-d8966e66c3d4 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:29.328152 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:29.328298 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0c6afc-fbc9-4f6a-ad3a-d8966e66c3d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:29.345295 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-fc0c6afc-fbc9-4f6a-ad3a-d8966e66c3d4 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:27:29.346554 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-fc0c6afc-fbc9-4f6a-ad3a-d8966e66c3d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:27:29.348544 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:29.369554 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a25241f-5bd9-407f-9477-9b8a78535fca None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:29.402441 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfc88c7-c925-4b64-9d8e-5396cf783305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:29.403150 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfc88c7-c925-4b64-9d8e-5396cf783305 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:29.403435 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfc88c7-c925-4b64-9d8e-5396cf783305 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:29.403718 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfc88c7-c925-4b64-9d8e-5396cf783305 None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:29.404979 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfc88c7-c925-4b64-9d8e-5396cf783305 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:27:29.405287 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfc88c7-c925-4b64-9d8e-5396cf783305 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:29.406641 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfc88c7-c925-4b64-9d8e-5396cf783305 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:29.419278 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 543/1089] 10.222.0.81 () {56 vars in 1115 bytes} [Wed Jul 20 22:27:29 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 22:27:29.419418 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 547/1090] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:27:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 22:27:29.420733 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfc88c7-c925-4b64-9d8e-5396cf783305 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:29.439546 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0c6afc-fbc9-4f6a-ad3a-d8966e66c3d4 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:29.522495 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfc88c7-c925-4b64-9d8e-5396cf783305 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:29.541872 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:29.555726 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0c6afc-fbc9-4f6a-ad3a-d8966e66c3d4 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:29.568355 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e4a327-6400-409e-ac70-f56558b8545a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:29.568355 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e4a327-6400-409e-ac70-f56558b8545a None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:29.568355 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e4a327-6400-409e-ac70-f56558b8545a None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:29.568355 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e4a327-6400-409e-ac70-f56558b8545a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:29.568776 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e4a327-6400-409e-ac70-f56558b8545a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:29.570099 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e4a327-6400-409e-ac70-f56558b8545a None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:29.588512 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-b3e4a327-6400-409e-ac70-f56558b8545a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:27:29.588512 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:27:29.588512 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:27:29.588512 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:27:29.588512 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:27:29.588512 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:27:29.588512 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:27:29.588512 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:27:29.588512 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:27:29.588512 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:27:29.588512 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:27:29.588512 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:27:29.588512 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 22:27:29.588512 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 22:27:29.588512 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 22:27:29.589505 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 22:27:29.589505 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:27:29.589505 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:27:29.589505 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:27:29.589505 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:27:29.589505 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:27:29.589505 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:27:29.589505 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:27:29.589505 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:27:29.589505 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:27:29.589505 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:27:29.589505 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:27:29.589505 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:27:29.589505 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:27:29.589505 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:27:29.589505 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:27:29.589505 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:27:29.590435 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 22:27:29.590435 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 22:27:29.590435 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:27:29.590435 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:27:29.595171 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 544/1091] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Jul 20 22:27:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 22:27:29.596091 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e4a327-6400-409e-ac70-f56558b8545a None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:29.596969 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c761d0f8-96f3-443b-b062-9c925d7a44b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:29.597787 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c761d0f8-96f3-443b-b062-9c925d7a44b4 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:29.598073 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c761d0f8-96f3-443b-b062-9c925d7a44b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:29.598349 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c761d0f8-96f3-443b-b062-9c925d7a44b4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:29.605382 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:29.609226 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c761d0f8-96f3-443b-b062-9c925d7a44b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 22:27:29.617360 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-c761d0f8-96f3-443b-b062-9c925d7a44b4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 22:27:29.617360 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:27:29.617360 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:27:29.617360 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:27:29.617360 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:27:29.617360 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:27:29.617360 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:27:29.617360 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:27:29.617360 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:27:29.617360 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:27:29.617360 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:27:29.617360 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:27:29.617360 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 22:27:29.617360 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 22:27:29.617360 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 22:27:29.617360 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 22:27:29.621212 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 22:27:29.621212 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 22:27:29.621212 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 22:27:29.621212 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 22:27:29.621212 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:27:29.621212 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:27:29.621212 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:27:29.621212 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:27:29.621212 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:27:29.621212 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:27:29.621212 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:27:29.621212 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:27:29.621212 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:27:29.621212 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:27:29.621212 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:27:29.621212 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:27:29.622070 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:27:29.622070 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:27:29.622070 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:27:29.622070 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:27:29.622070 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 22:27:29.622070 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 22:27:29.622070 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 22:27:29.622070 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 22:27:29.622070 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 22:27:29.622070 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 22:27:29.622070 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 22:27:29.622070 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:27:29.622070 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 548/1092] 10.222.0.81 () {56 vars in 1096 bytes} [Wed Jul 20 22:27:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 22:27:29.625022 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c761d0f8-96f3-443b-b062-9c925d7a44b4 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:29.697529 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e4a327-6400-409e-ac70-f56558b8545a None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:29.724629 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c761d0f8-96f3-443b-b062-9c925d7a44b4 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:29.730302 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f66a1b4-f264-489d-a9de-6b7f9b965f1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:29.730970 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f66a1b4-f264-489d-a9de-6b7f9b965f1f None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:29.731292 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f66a1b4-f264-489d-a9de-6b7f9b965f1f None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:29.731582 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f66a1b4-f264-489d-a9de-6b7f9b965f1f None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:29.732300 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f66a1b4-f264-489d-a9de-6b7f9b965f1f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:27:29.733125 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f66a1b4-f264-489d-a9de-6b7f9b965f1f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:29.734516 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f66a1b4-f264-489d-a9de-6b7f9b965f1f None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:29.745961 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 545/1093] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Jul 20 22:27:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:27:29.752211 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f66a1b4-f264-489d-a9de-6b7f9b965f1f None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:29.758571 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37be09f-2185-443c-9b5f-fd58bcf2d14f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:29.759261 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37be09f-2185-443c-9b5f-fd58bcf2d14f None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:29.759552 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37be09f-2185-443c-9b5f-fd58bcf2d14f None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:29.759833 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37be09f-2185-443c-9b5f-fd58bcf2d14f None admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:29.760573 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37be09f-2185-443c-9b5f-fd58bcf2d14f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:27:29.760882 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37be09f-2185-443c-9b5f-fd58bcf2d14f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:29.762238 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37be09f-2185-443c-9b5f-fd58bcf2d14f None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:29.775107 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 549/1094] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Jul 20 22:27:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 22:27:29.776054 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37be09f-2185-443c-9b5f-fd58bcf2d14f None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:29.785187 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:29.836833 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:29.860280 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f66a1b4-f264-489d-a9de-6b7f9b965f1f None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:29.879336 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37be09f-2185-443c-9b5f-fd58bcf2d14f None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:29.894690 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb2e91e-b1e1-4b4a-957a-e42784bae115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:29.895559 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb2e91e-b1e1-4b4a-957a-e42784bae115 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:29.895842 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb2e91e-b1e1-4b4a-957a-e42784bae115 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:29.896118 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb2e91e-b1e1-4b4a-957a-e42784bae115 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:29.903359 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb2e91e-b1e1-4b4a-957a-e42784bae115 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:29.904909 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb2e91e-b1e1-4b4a-957a-e42784bae115 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:29.913920 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cf2619-cb12-4808-8ac0-605beed25cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:29.914805 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cf2619-cb12-4808-8ac0-605beed25cd0 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:29.915127 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cf2619-cb12-4808-8ac0-605beed25cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:29.915409 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cf2619-cb12-4808-8ac0-605beed25cd0 None admin] PATH_INFO: `/v3/users/169f6a90d744404ab6970d012e804eb5` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:29.916603 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cf2619-cb12-4808-8ac0-605beed25cd0 None admin] RBAC: Authorizing `identity:get_user(user_id=169f6a90d744404ab6970d012e804eb5)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:29.918015 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cf2619-cb12-4808-8ac0-605beed25cd0 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:29.919698 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 550/1095] 10.222.0.81 () {56 vars in 1177 bytes} [Wed Jul 20 22:27:29 2022] GET /identity/v3/users/169f6a90d744404ab6970d012e804eb5 => generated 285 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:27:29.920745 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-2bb2e91e-b1e1-4b4a-957a-e42784bae115 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 20 22:27:29.920745 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:27:29.920745 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:27:29.920745 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:27:29.920745 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:27:29.920745 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:27:29.920745 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:27:29.920745 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:27:29.920745 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:27:29.920745 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:27:29.920745 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:27:29.920745 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:27:29.920745 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 22:27:29.920745 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 22:27:29.920745 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 22:27:29.920745 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 22:27:29.921888 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:27:29.921888 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:27:29.921888 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 22:27:29.921888 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 22:27:29.921888 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 22:27:29.921888 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 22:27:29.921888 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:27:29.921888 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:27:29.921888 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:27:29.921888 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:27:29.921888 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:27:29.921888 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:27:29.921888 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:27:29.921888 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:27:29.921888 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:27:29.921888 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:27:29.921888 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:27:29.921888 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:27:29.925625 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:27:29.925625 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:27:29.925625 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 22:27:29.925625 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 22:27:29.925625 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 22:27:29.925625 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 22:27:29.925625 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 22:27:29.925625 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 22:27:29.925625 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 20 22:27:29.925625 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:27:29.925625 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 546/1096] 10.222.0.81 () {56 vars in 1093 bytes} [Wed Jul 20 22:27:29 2022] GET /identity/v3/users/nova => generated 82 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 22:27:29.927676 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb2e91e-b1e1-4b4a-957a-e42784bae115 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:29.927908 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cf2619-cb12-4808-8ac0-605beed25cd0 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:30.019609 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:30.028173 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb2e91e-b1e1-4b4a-957a-e42784bae115 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:30.028314 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cf2619-cb12-4808-8ac0-605beed25cd0 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:30.055715 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:30.062330 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dbd721-68d4-41b7-9bf4-668ff1e57bf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:30.063218 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dbd721-68d4-41b7-9bf4-668ff1e57bf5 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:30.063506 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dbd721-68d4-41b7-9bf4-668ff1e57bf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:30.063612 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4969c7b2-e65f-4e13-8d51-60bba2253105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:30.063783 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dbd721-68d4-41b7-9bf4-668ff1e57bf5 None admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:30.064178 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4969c7b2-e65f-4e13-8d51-60bba2253105 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:30.064467 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4969c7b2-e65f-4e13-8d51-60bba2253105 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:30.064616 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dbd721-68d4-41b7-9bf4-668ff1e57bf5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:27:30.064748 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4969c7b2-e65f-4e13-8d51-60bba2253105 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:30.064930 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dbd721-68d4-41b7-9bf4-668ff1e57bf5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:30.066352 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dbd721-68d4-41b7-9bf4-668ff1e57bf5 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:30.085855 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 547/1097] 10.222.0.81 () {56 vars in 1133 bytes} [Wed Jul 20 22:27:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:27:30.091469 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dbd721-68d4-41b7-9bf4-668ff1e57bf5 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:30.133329 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4969c7b2-e65f-4e13-8d51-60bba2253105 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:30.134839 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4969c7b2-e65f-4e13-8d51-60bba2253105 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:30.190958 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dbd721-68d4-41b7-9bf4-668ff1e57bf5 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:30.226296 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91faa492-f945-494b-9c64-a4ef3a1f613f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:30.226956 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91faa492-f945-494b-9c64-a4ef3a1f613f None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:30.227271 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91faa492-f945-494b-9c64-a4ef3a1f613f None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:30.227552 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91faa492-f945-494b-9c64-a4ef3a1f613f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:30.235873 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 551/1098] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Jul 20 22:27:29 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 22:27:30.241652 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4969c7b2-e65f-4e13-8d51-60bba2253105 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:30.249257 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:30.268582 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:30.297299 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91faa492-f945-494b-9c64-a4ef3a1f613f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:30.298720 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91faa492-f945-494b-9c64-a4ef3a1f613f None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:30.339118 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4969c7b2-e65f-4e13-8d51-60bba2253105 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:30.372806 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa26215-5d7a-4860-bc05-6f76ce3ea921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:30.374190 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa26215-5d7a-4860-bc05-6f76ce3ea921 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:30.374466 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa26215-5d7a-4860-bc05-6f76ce3ea921 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:30.374741 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa26215-5d7a-4860-bc05-6f76ce3ea921 None admin] PATH_INFO: `/v3/projects/1db282486e4e45188be54680737fd949` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:30.375935 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa26215-5d7a-4860-bc05-6f76ce3ea921 None admin] RBAC: Authorizing `identity:get_project(project_id=1db282486e4e45188be54680737fd949)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:30.377357 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa26215-5d7a-4860-bc05-6f76ce3ea921 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:30.384009 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 552/1099] 10.222.0.81 () {56 vars in 1186 bytes} [Wed Jul 20 22:27:30 2022] GET /identity/v3/projects/1db282486e4e45188be54680737fd949 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 22:27:30.385209 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa26215-5d7a-4860-bc05-6f76ce3ea921 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:30.396065 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 548/1100] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Jul 20 22:27:30 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 20 22:27:30.401578 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91faa492-f945-494b-9c64-a4ef3a1f613f None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:30.449122 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:30.477156 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:30.483296 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa26215-5d7a-4860-bc05-6f76ce3ea921 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:30.498048 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91faa492-f945-494b-9c64-a4ef3a1f613f None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:30.517216 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68b807d-9fad-4e82-bbf2-9a49699829ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:30.517874 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68b807d-9fad-4e82-bbf2-9a49699829ec None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:30.518162 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68b807d-9fad-4e82-bbf2-9a49699829ec None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:30.518452 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68b807d-9fad-4e82-bbf2-9a49699829ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:30.519308 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68b807d-9fad-4e82-bbf2-9a49699829ec None admin] RBAC: Adding query filter params (role.id=d89b6c92d35a401a8517f1d47f6be843, scope.project.id=1db282486e4e45188be54680737fd949, user.id=169f6a90d744404ab6970d012e804eb5) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:27:30.519626 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68b807d-9fad-4e82-bbf2-9a49699829ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:30.521060 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68b807d-9fad-4e82-bbf2-9a49699829ec None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:30.527074 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 553/1101] 10.222.0.81 () {56 vars in 1374 bytes} [Wed Jul 20 22:27:30 2022] GET /identity/v3/role_assignments?user.id=169f6a90d744404ab6970d012e804eb5&scope.project.id=1db282486e4e45188be54680737fd949&role.id=d89b6c92d35a401a8517f1d47f6be843 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 22:27:30.537396 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf19121-5b1a-4f5f-8df6-fc9d77158d98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:30.538336 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf19121-5b1a-4f5f-8df6-fc9d77158d98 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:30.538695 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf19121-5b1a-4f5f-8df6-fc9d77158d98 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:30.539076 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf19121-5b1a-4f5f-8df6-fc9d77158d98 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:30.551305 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf19121-5b1a-4f5f-8df6-fc9d77158d98 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:30.552704 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf19121-5b1a-4f5f-8df6-fc9d77158d98 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:30.572925 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-3cf19121-5b1a-4f5f-8df6-fc9d77158d98 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:27:30.572925 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:27:30.572925 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:27:30.572925 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:27:30.572925 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:27:30.572925 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:27:30.572925 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:27:30.572925 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:27:30.572925 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:27:30.572925 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:27:30.572925 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:27:30.572925 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:27:30.572925 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 22:27:30.572925 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 22:27:30.572925 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 22:27:30.573911 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 22:27:30.573911 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:27:30.573911 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:27:30.573911 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:27:30.573911 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:27:30.573911 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:27:30.573911 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:27:30.573911 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:27:30.573911 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:27:30.573911 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:27:30.573911 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:27:30.573911 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:27:30.573911 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:27:30.573911 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:27:30.573911 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:27:30.573911 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:27:30.573911 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:27:30.575040 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 22:27:30.575040 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 22:27:30.575040 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:27:30.575040 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:27:30.575040 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 549/1102] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Jul 20 22:27:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 22:27:30.580445 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68b807d-9fad-4e82-bbf2-9a49699829ec None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:30.688647 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68b807d-9fad-4e82-bbf2-9a49699829ec None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:30.721924 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8a0ef1-b027-41c4-9cfe-adb766c57cb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:30.722608 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8a0ef1-b027-41c4-9cfe-adb766c57cb2 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:30.722913 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8a0ef1-b027-41c4-9cfe-adb766c57cb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:30.723222 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8a0ef1-b027-41c4-9cfe-adb766c57cb2 None admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:30.723956 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8a0ef1-b027-41c4-9cfe-adb766c57cb2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:27:30.724282 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8a0ef1-b027-41c4-9cfe-adb766c57cb2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:30.725663 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8a0ef1-b027-41c4-9cfe-adb766c57cb2 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:30.738099 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 554/1103] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Jul 20 22:27:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:27:30.743843 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf19121-5b1a-4f5f-8df6-fc9d77158d98 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:30.840782 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf19121-5b1a-4f5f-8df6-fc9d77158d98 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:30.842306 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:30.875383 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca8a3c1-ca37-45d8-a958-ccf968f09df8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:30.876197 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca8a3c1-ca37-45d8-a958-ccf968f09df8 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:30.876499 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca8a3c1-ca37-45d8-a958-ccf968f09df8 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:30.876794 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca8a3c1-ca37-45d8-a958-ccf968f09df8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:30.887421 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca8a3c1-ca37-45d8-a958-ccf968f09df8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:30.888945 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca8a3c1-ca37-45d8-a958-ccf968f09df8 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:30.907043 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-8ca8a3c1-ca37-45d8-a958-ccf968f09df8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 22:27:30.907043 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:27:30.907043 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:27:30.907043 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:27:30.907043 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:27:30.907043 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:27:30.907043 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:27:30.907043 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:27:30.907043 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:27:30.907043 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:27:30.907043 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:27:30.907043 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:27:30.907043 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 22:27:30.907043 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 22:27:30.907043 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 22:27:30.908189 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 22:27:30.908189 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:27:30.908189 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:27:30.908189 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:27:30.908189 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:27:30.908189 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:27:30.908189 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:27:30.908189 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:27:30.908189 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:27:30.908189 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:27:30.908189 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:27:30.908189 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:27:30.908189 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:27:30.908189 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:27:30.908189 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:27:30.908189 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:27:30.908189 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:27:30.909412 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 22:27:30.909412 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 22:27:30.909412 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 22:27:30.909412 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 22:27:30.909412 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 22:27:30.909412 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 22:27:30.909412 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 22:27:30.909412 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:27:30.909412 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 550/1104] 10.222.0.81 () {56 vars in 1111 bytes} [Wed Jul 20 22:27:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:27:30.914330 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8a0ef1-b027-41c4-9cfe-adb766c57cb2 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:30.936505 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:31.015145 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8a0ef1-b027-41c4-9cfe-adb766c57cb2 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:31.049416 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e297f9e9-43ee-4c2c-9a0a-1d8f74529167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:31.050090 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e297f9e9-43ee-4c2c-9a0a-1d8f74529167 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:31.050375 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e297f9e9-43ee-4c2c-9a0a-1d8f74529167 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:31.050656 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e297f9e9-43ee-4c2c-9a0a-1d8f74529167 None admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:31.051482 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e297f9e9-43ee-4c2c-9a0a-1d8f74529167 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:27:31.051894 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e297f9e9-43ee-4c2c-9a0a-1d8f74529167 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:31.053327 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e297f9e9-43ee-4c2c-9a0a-1d8f74529167 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:31.070475 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 555/1105] 10.222.0.81 () {56 vars in 1148 bytes} [Wed Jul 20 22:27:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:27:31.071696 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca8a3c1-ca37-45d8-a958-ccf968f09df8 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:31.172970 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca8a3c1-ca37-45d8-a958-ccf968f09df8 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:31.213229 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2f83af-c0da-4416-8796-36321be58c6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:31.213896 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2f83af-c0da-4416-8796-36321be58c6d None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:31.214183 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2f83af-c0da-4416-8796-36321be58c6d None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:31.214462 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2f83af-c0da-4416-8796-36321be58c6d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:31.215317 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2f83af-c0da-4416-8796-36321be58c6d None admin] RBAC: Adding query filter params (role.id=a3e411d61a234343b961e8bf78a55a11, scope.project.id=385edf2e59e546d090ace36600e29032, user.id=7829324db2574d38b18c749d0559177f) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:27:31.215633 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2f83af-c0da-4416-8796-36321be58c6d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:31.217097 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2f83af-c0da-4416-8796-36321be58c6d None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:31.222908 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 551/1106] 10.222.0.81 () {56 vars in 1374 bytes} [Wed Jul 20 22:27:31 2022] GET /identity/v3/role_assignments?user.id=7829324db2574d38b18c749d0559177f&scope.project.id=385edf2e59e546d090ace36600e29032&role.id=a3e411d61a234343b961e8bf78a55a11 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:27:32.597335 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafce43c-2f84-45ff-890d-b7e87504994e None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:32.597872 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafce43c-2f84-45ff-890d-b7e87504994e None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:32.598040 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafce43c-2f84-45ff-890d-b7e87504994e None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:32.599644 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 556/1107] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:27:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:27:32.604846 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ed5d32-8cea-4a38-a139-fea1bc205fd0 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:32.605269 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ed5d32-8cea-4a38-a139-fea1bc205fd0 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:32.605411 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ed5d32-8cea-4a38-a139-fea1bc205fd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:32.622282 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-54ed5d32-8cea-4a38-a139-fea1bc205fd0 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:27:32.623582 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-54ed5d32-8cea-4a38-a139-fea1bc205fd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:27:32.642947 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:32.688281 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 552/1108] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:27:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 22:27:32.712326 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafce43c-2f84-45ff-890d-b7e87504994e None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:32.792281 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:32.811279 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafce43c-2f84-45ff-890d-b7e87504994e None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:32.845089 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2d5491-2d5d-4c5b-8980-28d890d83c67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:32.845872 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2d5491-2d5d-4c5b-8980-28d890d83c67 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:32.846150 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2d5491-2d5d-4c5b-8980-28d890d83c67 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:32.846423 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2d5491-2d5d-4c5b-8980-28d890d83c67 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:32.847697 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2d5491-2d5d-4c5b-8980-28d890d83c67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:32.849093 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2d5491-2d5d-4c5b-8980-28d890d83c67 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:32.850671 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 557/1109] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Jul 20 22:27:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:27:32.856368 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ed5d32-8cea-4a38-a139-fea1bc205fd0 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:32.952061 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ed5d32-8cea-4a38-a139-fea1bc205fd0 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:32.984042 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ff3a42-5fb4-425b-ac10-768091ebe386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:32.984705 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ff3a42-5fb4-425b-ac10-768091ebe386 None admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:32.984989 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ff3a42-5fb4-425b-ac10-768091ebe386 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:32.985266 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ff3a42-5fb4-425b-ac10-768091ebe386 None admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:32.985966 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ff3a42-5fb4-425b-ac10-768091ebe386 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:32.987446 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ff3a42-5fb4-425b-ac10-768091ebe386 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:33.083288 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 553/1110] 10.222.0.81 () {60 vars in 1132 bytes} [Wed Jul 20 22:27:32 2022] POST /identity/v3/users => generated 286 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:27:35.140394 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162f3e37-bb32-45a8-a602-9eef2b290cae None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:35.140967 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162f3e37-bb32-45a8-a602-9eef2b290cae None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:35.140967 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162f3e37-bb32-45a8-a602-9eef2b290cae None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:35.142338 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 558/1111] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:27:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:27:35.147628 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c4c489-a019-4159-b8da-ccf25362a472 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:35.148048 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c4c489-a019-4159-b8da-ccf25362a472 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:35.148197 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c4c489-a019-4159-b8da-ccf25362a472 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:35.165680 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-e9c4c489-a019-4159-b8da-ccf25362a472 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:27:35.166885 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-e9c4c489-a019-4159-b8da-ccf25362a472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:27:35.174594 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:35.236939 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 554/1112] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:27:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 22:27:35.260260 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162f3e37-bb32-45a8-a602-9eef2b290cae None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:35.359866 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162f3e37-bb32-45a8-a602-9eef2b290cae None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:35.393762 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6e21e7-38ca-4172-ae8d-a9e7a40516c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:35.394559 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6e21e7-38ca-4172-ae8d-a9e7a40516c1 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:35.394846 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6e21e7-38ca-4172-ae8d-a9e7a40516c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:35.395157 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6e21e7-38ca-4172-ae8d-a9e7a40516c1 None admin] PATH_INFO: `/v3/roles/6d5a3e6a22fa4f4fa47afcfce0246a1f` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:35.396616 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6e21e7-38ca-4172-ae8d-a9e7a40516c1 None admin] RBAC: Authorizing `identity:get_role(role_id=6d5a3e6a22fa4f4fa47afcfce0246a1f)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:35.397972 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6e21e7-38ca-4172-ae8d-a9e7a40516c1 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:35.399346 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 559/1113] 10.222.0.81 () {56 vars in 1177 bytes} [Wed Jul 20 22:27:35 2022] GET /identity/v3/roles/6d5a3e6a22fa4f4fa47afcfce0246a1f => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:27:35.405074 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c4c489-a019-4159-b8da-ccf25362a472 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:35.502913 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c4c489-a019-4159-b8da-ccf25362a472 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:35.540717 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37320d2-9282-4c2b-bb52-8151f297a201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:35.542271 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37320d2-9282-4c2b-bb52-8151f297a201 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:35.542701 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37320d2-9282-4c2b-bb52-8151f297a201 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:35.543091 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37320d2-9282-4c2b-bb52-8151f297a201 None admin] PATH_INFO: `/v3/users/f930e1e75fc546df94d3d5ed79a196b9` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:35.557194 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:35.560771 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37320d2-9282-4c2b-bb52-8151f297a201 None admin] RBAC: Authorizing `identity:get_user(user_id=f930e1e75fc546df94d3d5ed79a196b9)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:35.562189 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37320d2-9282-4c2b-bb52-8151f297a201 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:35.563913 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 555/1114] 10.222.0.81 () {56 vars in 1177 bytes} [Wed Jul 20 22:27:35 2022] GET /identity/v3/users/f930e1e75fc546df94d3d5ed79a196b9 => generated 286 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:27:35.569721 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6e21e7-38ca-4172-ae8d-a9e7a40516c1 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:35.588228 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:35.668478 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6e21e7-38ca-4172-ae8d-a9e7a40516c1 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:35.701210 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd56cff1-45a6-4cf4-8b58-ec0164dbaef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:35.701848 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd56cff1-45a6-4cf4-8b58-ec0164dbaef0 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:35.702136 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd56cff1-45a6-4cf4-8b58-ec0164dbaef0 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:35.702418 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd56cff1-45a6-4cf4-8b58-ec0164dbaef0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:35.767769 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd56cff1-45a6-4cf4-8b58-ec0164dbaef0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:35.769224 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd56cff1-45a6-4cf4-8b58-ec0164dbaef0 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:35.786026 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:35.867766 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 560/1115] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Jul 20 22:27:35 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 22:27:35.873714 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37320d2-9282-4c2b-bb52-8151f297a201 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:36.049594 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37320d2-9282-4c2b-bb52-8151f297a201 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:36.082196 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ef4eb9-08a2-4df3-a045-8a2445f6c8e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:36.083134 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ef4eb9-08a2-4df3-a045-8a2445f6c8e1 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:36.083424 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ef4eb9-08a2-4df3-a045-8a2445f6c8e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:36.083711 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ef4eb9-08a2-4df3-a045-8a2445f6c8e1 None admin] PATH_INFO: `/v3/projects/1db282486e4e45188be54680737fd949` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:36.084894 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ef4eb9-08a2-4df3-a045-8a2445f6c8e1 None admin] RBAC: Authorizing `identity:get_project(project_id=1db282486e4e45188be54680737fd949)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:36.086323 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ef4eb9-08a2-4df3-a045-8a2445f6c8e1 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:36.088066 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 556/1116] 10.222.0.81 () {56 vars in 1186 bytes} [Wed Jul 20 22:27:35 2022] GET /identity/v3/projects/1db282486e4e45188be54680737fd949 => generated 311 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:27:36.094825 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd56cff1-45a6-4cf4-8b58-ec0164dbaef0 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:36.191895 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd56cff1-45a6-4cf4-8b58-ec0164dbaef0 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:36.208371 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:36.225172 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed222d0a-aca0-4d6b-9ed2-f732a169a511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:36.225823 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed222d0a-aca0-4d6b-9ed2-f732a169a511 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:36.226107 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed222d0a-aca0-4d6b-9ed2-f732a169a511 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:36.226383 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed222d0a-aca0-4d6b-9ed2-f732a169a511 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:36.227232 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed222d0a-aca0-4d6b-9ed2-f732a169a511 None admin] RBAC: Adding query filter params (role.id=6d5a3e6a22fa4f4fa47afcfce0246a1f, scope.project.id=1db282486e4e45188be54680737fd949, user.id=f930e1e75fc546df94d3d5ed79a196b9) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:27:36.227545 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed222d0a-aca0-4d6b-9ed2-f732a169a511 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:36.228958 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed222d0a-aca0-4d6b-9ed2-f732a169a511 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:36.234250 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 561/1117] 10.222.0.81 () {56 vars in 1374 bytes} [Wed Jul 20 22:27:36 2022] GET /identity/v3/role_assignments?user.id=f930e1e75fc546df94d3d5ed79a196b9&scope.project.id=1db282486e4e45188be54680737fd949&role.id=6d5a3e6a22fa4f4fa47afcfce0246a1f => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:27:38.231479 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e274dd90-a120-4af7-a428-8f59b81c6f33 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:38.231939 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e274dd90-a120-4af7-a428-8f59b81c6f33 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:38.232155 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e274dd90-a120-4af7-a428-8f59b81c6f33 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:38.233537 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 557/1118] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:27:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:27:38.238999 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af15a12-019d-4a9f-82cf-02445789787c None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:38.239429 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af15a12-019d-4a9f-82cf-02445789787c None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:38.239593 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af15a12-019d-4a9f-82cf-02445789787c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:38.255935 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-7af15a12-019d-4a9f-82cf-02445789787c None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:27:38.257121 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-7af15a12-019d-4a9f-82cf-02445789787c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:27:38.322136 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 562/1119] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:27:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 22:27:38.345683 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e274dd90-a120-4af7-a428-8f59b81c6f33 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:38.411704 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:38.444366 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e274dd90-a120-4af7-a428-8f59b81c6f33 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:38.477954 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a20a07-c47c-42ec-8902-6a9eb964787f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:38.478850 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a20a07-c47c-42ec-8902-6a9eb964787f None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:38.479168 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a20a07-c47c-42ec-8902-6a9eb964787f None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:38.479445 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a20a07-c47c-42ec-8902-6a9eb964787f None admin] PATH_INFO: `/v3/roles/6d5a3e6a22fa4f4fa47afcfce0246a1f` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:38.480947 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a20a07-c47c-42ec-8902-6a9eb964787f None admin] RBAC: Authorizing `identity:get_role(role_id=6d5a3e6a22fa4f4fa47afcfce0246a1f)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:38.482313 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a20a07-c47c-42ec-8902-6a9eb964787f None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:38.483709 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 558/1120] 10.222.0.81 () {56 vars in 1177 bytes} [Wed Jul 20 22:27:38 2022] GET /identity/v3/roles/6d5a3e6a22fa4f4fa47afcfce0246a1f => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:27:38.489603 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af15a12-019d-4a9f-82cf-02445789787c None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:38.586344 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af15a12-019d-4a9f-82cf-02445789787c None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:38.596699 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:38.619120 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4872c5-9a7c-4b24-801c-4b213160aaf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:38.620008 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4872c5-9a7c-4b24-801c-4b213160aaf6 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:38.620294 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4872c5-9a7c-4b24-801c-4b213160aaf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:38.620575 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4872c5-9a7c-4b24-801c-4b213160aaf6 None admin] PATH_INFO: `/v3/users/f930e1e75fc546df94d3d5ed79a196b9` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:38.621753 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4872c5-9a7c-4b24-801c-4b213160aaf6 None admin] RBAC: Authorizing `identity:get_user(user_id=f930e1e75fc546df94d3d5ed79a196b9)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:38.623765 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4872c5-9a7c-4b24-801c-4b213160aaf6 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:38.625464 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 563/1121] 10.222.0.81 () {56 vars in 1177 bytes} [Wed Jul 20 22:27:38 2022] GET /identity/v3/users/f930e1e75fc546df94d3d5ed79a196b9 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:27:38.631272 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a20a07-c47c-42ec-8902-6a9eb964787f None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:38.727496 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a20a07-c47c-42ec-8902-6a9eb964787f None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:38.760122 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdac686e-6f4b-48b1-b600-fdcc3df6427c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:38.760979 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdac686e-6f4b-48b1-b600-fdcc3df6427c None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:38.761285 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdac686e-6f4b-48b1-b600-fdcc3df6427c None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:38.761558 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdac686e-6f4b-48b1-b600-fdcc3df6427c None admin] PATH_INFO: `/v3/projects/1db282486e4e45188be54680737fd949` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:38.763229 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdac686e-6f4b-48b1-b600-fdcc3df6427c None admin] RBAC: Authorizing `identity:get_project(project_id=1db282486e4e45188be54680737fd949)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:38.764668 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdac686e-6f4b-48b1-b600-fdcc3df6427c None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:38.766351 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 559/1122] 10.222.0.81 () {56 vars in 1186 bytes} [Wed Jul 20 22:27:38 2022] GET /identity/v3/projects/1db282486e4e45188be54680737fd949 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:27:38.772206 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4872c5-9a7c-4b24-801c-4b213160aaf6 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:38.871143 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4872c5-9a7c-4b24-801c-4b213160aaf6 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:38.905254 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30551833-ad2d-4f5e-a694-042bf9f80185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:38.906134 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30551833-ad2d-4f5e-a694-042bf9f80185 None admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:38.906422 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30551833-ad2d-4f5e-a694-042bf9f80185 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:38.906701 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30551833-ad2d-4f5e-a694-042bf9f80185 None admin] PATH_INFO: `/v3/projects/1db282486e4e45188be54680737fd949/users/f930e1e75fc546df94d3d5ed79a196b9/roles/6d5a3e6a22fa4f4fa47afcfce0246a1f` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:38.909164 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30551833-ad2d-4f5e-a694-042bf9f80185 None admin] RBAC: Authorizing `identity:create_grant(project_id=1db282486e4e45188be54680737fd949, user_id=f930e1e75fc546df94d3d5ed79a196b9, role_id=6d5a3e6a22fa4f4fa47afcfce0246a1f)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:38.910611 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30551833-ad2d-4f5e-a694-042bf9f80185 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:39.014055 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 564/1123] 10.222.0.81 () {58 vars in 1439 bytes} [Wed Jul 20 22:27:38 2022] PUT /identity/v3/projects/1db282486e4e45188be54680737fd949/users/f930e1e75fc546df94d3d5ed79a196b9/roles/6d5a3e6a22fa4f4fa47afcfce0246a1f => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:27:41.148291 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64e2a02-b5ae-4abf-a480-c7a899302828 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:41.148826 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64e2a02-b5ae-4abf-a480-c7a899302828 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:41.148826 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64e2a02-b5ae-4abf-a480-c7a899302828 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:41.150177 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 560/1124] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:27:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:27:41.156251 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b516f981-2bc7-499f-a95b-99c1c91bf9d8 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:41.156632 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b516f981-2bc7-499f-a95b-99c1c91bf9d8 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:41.156829 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b516f981-2bc7-499f-a95b-99c1c91bf9d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:41.174016 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-b516f981-2bc7-499f-a95b-99c1c91bf9d8 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:27:41.175239 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-b516f981-2bc7-499f-a95b-99c1c91bf9d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:27:41.204470 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:41.240888 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 565/1125] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:27:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 22:27:41.264743 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64e2a02-b5ae-4abf-a480-c7a899302828 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:41.268521 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:41.367071 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64e2a02-b5ae-4abf-a480-c7a899302828 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:41.400874 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bff2ba1-130a-4c1f-9334-331983386b01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:41.401675 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bff2ba1-130a-4c1f-9334-331983386b01 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:41.401963 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bff2ba1-130a-4c1f-9334-331983386b01 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:41.402241 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bff2ba1-130a-4c1f-9334-331983386b01 None admin] PATH_INFO: `/v3/roles/6d5a3e6a22fa4f4fa47afcfce0246a1f` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:41.404271 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bff2ba1-130a-4c1f-9334-331983386b01 None admin] RBAC: Authorizing `identity:get_role(role_id=6d5a3e6a22fa4f4fa47afcfce0246a1f)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:41.405647 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bff2ba1-130a-4c1f-9334-331983386b01 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:41.407022 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 561/1126] 10.222.0.81 () {56 vars in 1177 bytes} [Wed Jul 20 22:27:41 2022] GET /identity/v3/roles/6d5a3e6a22fa4f4fa47afcfce0246a1f => generated 223 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:27:41.413128 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b516f981-2bc7-499f-a95b-99c1c91bf9d8 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:41.509459 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b516f981-2bc7-499f-a95b-99c1c91bf9d8 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:41.544939 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88421555-4b01-4365-ae38-15641ebd06da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:41.545831 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88421555-4b01-4365-ae38-15641ebd06da None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:41.546117 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88421555-4b01-4365-ae38-15641ebd06da None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:41.546397 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88421555-4b01-4365-ae38-15641ebd06da None admin] PATH_INFO: `/v3/users/f930e1e75fc546df94d3d5ed79a196b9` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:41.547604 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88421555-4b01-4365-ae38-15641ebd06da None admin] RBAC: Authorizing `identity:get_user(user_id=f930e1e75fc546df94d3d5ed79a196b9)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:41.549031 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88421555-4b01-4365-ae38-15641ebd06da None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:41.550640 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 566/1127] 10.222.0.81 () {56 vars in 1177 bytes} [Wed Jul 20 22:27:41 2022] GET /identity/v3/users/f930e1e75fc546df94d3d5ed79a196b9 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:27:41.556341 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bff2ba1-130a-4c1f-9334-331983386b01 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:41.624923 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:41.653915 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bff2ba1-130a-4c1f-9334-331983386b01 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:41.691159 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a117759a-12a5-48e9-bd45-e088a3fc1fd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:41.691814 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a117759a-12a5-48e9-bd45-e088a3fc1fd6 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:41.692101 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a117759a-12a5-48e9-bd45-e088a3fc1fd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:41.692382 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a117759a-12a5-48e9-bd45-e088a3fc1fd6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:41.763862 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a117759a-12a5-48e9-bd45-e088a3fc1fd6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:41.765306 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a117759a-12a5-48e9-bd45-e088a3fc1fd6 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:41.830987 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:41.860725 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 562/1128] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Jul 20 22:27:41 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 22:27:41.866383 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88421555-4b01-4365-ae38-15641ebd06da None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:41.900485 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:41.963200 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88421555-4b01-4365-ae38-15641ebd06da None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:41.997532 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d93d973-b2ea-4c8b-a78e-2db70649c389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:41.998410 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d93d973-b2ea-4c8b-a78e-2db70649c389 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:41.998695 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d93d973-b2ea-4c8b-a78e-2db70649c389 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:41.999034 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d93d973-b2ea-4c8b-a78e-2db70649c389 None admin] PATH_INFO: `/v3/projects/1db282486e4e45188be54680737fd949` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:42.000245 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d93d973-b2ea-4c8b-a78e-2db70649c389 None admin] RBAC: Authorizing `identity:get_project(project_id=1db282486e4e45188be54680737fd949)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:42.001695 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d93d973-b2ea-4c8b-a78e-2db70649c389 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:42.003456 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 567/1129] 10.222.0.81 () {56 vars in 1186 bytes} [Wed Jul 20 22:27:41 2022] GET /identity/v3/projects/1db282486e4e45188be54680737fd949 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:27:42.009925 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a117759a-12a5-48e9-bd45-e088a3fc1fd6 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:42.104415 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a117759a-12a5-48e9-bd45-e088a3fc1fd6 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:42.138098 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112e6ce7-bab2-4cde-b6d7-ca5a1876c593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:42.138743 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112e6ce7-bab2-4cde-b6d7-ca5a1876c593 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:42.139047 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112e6ce7-bab2-4cde-b6d7-ca5a1876c593 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:42.139329 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112e6ce7-bab2-4cde-b6d7-ca5a1876c593 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:42.140143 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112e6ce7-bab2-4cde-b6d7-ca5a1876c593 None admin] RBAC: Adding query filter params (role.id=6d5a3e6a22fa4f4fa47afcfce0246a1f, scope.project.id=1db282486e4e45188be54680737fd949, user.id=f930e1e75fc546df94d3d5ed79a196b9) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:27:42.140456 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112e6ce7-bab2-4cde-b6d7-ca5a1876c593 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:42.141874 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112e6ce7-bab2-4cde-b6d7-ca5a1876c593 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:42.147533 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 563/1130] 10.222.0.81 () {56 vars in 1374 bytes} [Wed Jul 20 22:27:42 2022] GET /identity/v3/role_assignments?user.id=f930e1e75fc546df94d3d5ed79a196b9&scope.project.id=1db282486e4e45188be54680737fd949&role.id=6d5a3e6a22fa4f4fa47afcfce0246a1f => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:27:44.193287 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43c99ad-da78-4c67-bf0d-f9fcba238a34 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:44.193813 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43c99ad-da78-4c67-bf0d-f9fcba238a34 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:44.193925 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43c99ad-da78-4c67-bf0d-f9fcba238a34 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:44.195241 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 568/1131] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:27:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:27:44.201002 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a266af6-86b6-47cb-9d49-0c9282187230 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:44.201443 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a266af6-86b6-47cb-9d49-0c9282187230 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:44.201546 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a266af6-86b6-47cb-9d49-0c9282187230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:44.218589 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-8a266af6-86b6-47cb-9d49-0c9282187230 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:27:44.219814 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-8a266af6-86b6-47cb-9d49-0c9282187230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:27:44.247053 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:44.284644 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 564/1132] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:27:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 22:27:44.307787 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43c99ad-da78-4c67-bf0d-f9fcba238a34 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:44.408610 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43c99ad-da78-4c67-bf0d-f9fcba238a34 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:44.411116 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:44.442360 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6457b87e-d300-4f5f-bd0f-882f9fff0677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:44.443188 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6457b87e-d300-4f5f-bd0f-882f9fff0677 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:44.443595 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6457b87e-d300-4f5f-bd0f-882f9fff0677 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:44.443837 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6457b87e-d300-4f5f-bd0f-882f9fff0677 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:44.445018 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6457b87e-d300-4f5f-bd0f-882f9fff0677 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:44.446451 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6457b87e-d300-4f5f-bd0f-882f9fff0677 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:44.448483 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 569/1133] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Jul 20 22:27:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:27:44.454429 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a266af6-86b6-47cb-9d49-0c9282187230 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:44.550427 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a266af6-86b6-47cb-9d49-0c9282187230 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:44.584444 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3f77c4-61b0-4623-ae58-70c4d4d09fa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:44.585108 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3f77c4-61b0-4623-ae58-70c4d4d09fa9 None admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:44.585391 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3f77c4-61b0-4623-ae58-70c4d4d09fa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:44.585669 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3f77c4-61b0-4623-ae58-70c4d4d09fa9 None admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:44.586372 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3f77c4-61b0-4623-ae58-70c4d4d09fa9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:44.587818 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3f77c4-61b0-4623-ae58-70c4d4d09fa9 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:44.701332 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 565/1134] 10.222.0.81 () {60 vars in 1141 bytes} [Wed Jul 20 22:27:44 2022] POST /identity/v3/projects => generated 311 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:27:46.757132 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b5190e-7d6f-4bff-8cb8-f28e9881055f None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:46.757692 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b5190e-7d6f-4bff-8cb8-f28e9881055f None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:46.757692 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b5190e-7d6f-4bff-8cb8-f28e9881055f None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:46.759068 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 570/1135] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:27:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:27:46.764263 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63609cfd-733b-4918-ba7e-ad262c79905e None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:46.764653 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63609cfd-733b-4918-ba7e-ad262c79905e None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:46.764832 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63609cfd-733b-4918-ba7e-ad262c79905e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:46.781648 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-63609cfd-733b-4918-ba7e-ad262c79905e None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:27:46.783124 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-63609cfd-733b-4918-ba7e-ad262c79905e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:27:46.796563 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:46.847925 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 566/1136] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:27:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 22:27:46.873996 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b5190e-7d6f-4bff-8cb8-f28e9881055f None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:46.975796 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b5190e-7d6f-4bff-8cb8-f28e9881055f None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:47.008387 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d5f1ee-4289-4c2e-9187-612315800199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:47.009117 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d5f1ee-4289-4c2e-9187-612315800199 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:47.009399 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d5f1ee-4289-4c2e-9187-612315800199 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:47.009669 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d5f1ee-4289-4c2e-9187-612315800199 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:47.010827 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d5f1ee-4289-4c2e-9187-612315800199 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:47.012902 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d5f1ee-4289-4c2e-9187-612315800199 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:47.014511 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 571/1137] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Jul 20 22:27:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:27:47.020279 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63609cfd-733b-4918-ba7e-ad262c79905e None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:47.117611 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63609cfd-733b-4918-ba7e-ad262c79905e None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:47.150185 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a6e391-1004-4ddd-a10b-7d4048dd6e14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:47.150850 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a6e391-1004-4ddd-a10b-7d4048dd6e14 None admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:47.151186 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a6e391-1004-4ddd-a10b-7d4048dd6e14 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:47.151462 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a6e391-1004-4ddd-a10b-7d4048dd6e14 None admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:47.152167 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a6e391-1004-4ddd-a10b-7d4048dd6e14 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:47.154250 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a6e391-1004-4ddd-a10b-7d4048dd6e14 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:47.270932 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 567/1138] 10.222.0.81 () {60 vars in 1132 bytes} [Wed Jul 20 22:27:47 2022] POST /identity/v3/users => generated 286 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:27:49.319711 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4b2d09-fefe-4f5e-82d6-9b7abd0e7a2f None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:49.320219 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4b2d09-fefe-4f5e-82d6-9b7abd0e7a2f None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:49.320305 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4b2d09-fefe-4f5e-82d6-9b7abd0e7a2f None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:49.321633 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 572/1139] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:27:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:27:49.326706 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba39966f-e798-43e6-9fad-48e049767199 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:49.327161 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba39966f-e798-43e6-9fad-48e049767199 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:49.327300 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba39966f-e798-43e6-9fad-48e049767199 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:49.333166 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:49.343759 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-ba39966f-e798-43e6-9fad-48e049767199 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:27:49.344945 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-ba39966f-e798-43e6-9fad-48e049767199 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:27:49.411506 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 568/1140] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:27:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 22:27:49.435813 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4b2d09-fefe-4f5e-82d6-9b7abd0e7a2f None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:49.478010 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:49.535298 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4b2d09-fefe-4f5e-82d6-9b7abd0e7a2f None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:49.568849 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc88c555-28d4-4d51-8ff5-c7cf05bc231b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:49.569634 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc88c555-28d4-4d51-8ff5-c7cf05bc231b None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:49.569920 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc88c555-28d4-4d51-8ff5-c7cf05bc231b None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:49.570199 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc88c555-28d4-4d51-8ff5-c7cf05bc231b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:49.580186 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc88c555-28d4-4d51-8ff5-c7cf05bc231b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 22:27:49.588186 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-dc88c555-28d4-4d51-8ff5-c7cf05bc231b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 22:27:49.588186 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:27:49.588186 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:27:49.588186 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:27:49.588186 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:27:49.588186 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:27:49.588186 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:27:49.588186 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:27:49.588186 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:27:49.588186 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:27:49.588186 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:27:49.588186 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:27:49.588186 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 22:27:49.588186 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 22:27:49.588186 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 22:27:49.588186 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 22:27:49.589272 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 22:27:49.589272 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 22:27:49.589272 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 22:27:49.589272 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 22:27:49.589272 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:27:49.589272 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:27:49.589272 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:27:49.589272 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:27:49.589272 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:27:49.589272 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:27:49.589272 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:27:49.589272 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:27:49.589272 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:27:49.589272 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:27:49.589272 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:27:49.589272 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:27:49.590145 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:27:49.590145 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:27:49.590145 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:27:49.590145 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:27:49.590145 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 22:27:49.590145 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 22:27:49.590145 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 22:27:49.590145 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 22:27:49.590145 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 22:27:49.590145 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 22:27:49.590145 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 22:27:49.590145 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:27:49.590145 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 573/1141] 10.222.0.81 () {56 vars in 1096 bytes} [Wed Jul 20 22:27:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:27:49.595479 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba39966f-e798-43e6-9fad-48e049767199 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:49.693835 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba39966f-e798-43e6-9fad-48e049767199 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:49.727839 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bbe492-432e-43af-8284-03efb14577d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:49.728558 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bbe492-432e-43af-8284-03efb14577d3 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:49.728796 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bbe492-432e-43af-8284-03efb14577d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:49.729077 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bbe492-432e-43af-8284-03efb14577d3 None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:49.729813 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bbe492-432e-43af-8284-03efb14577d3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:27:49.730124 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bbe492-432e-43af-8284-03efb14577d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:49.731596 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bbe492-432e-43af-8284-03efb14577d3 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:49.732931 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:49.740099 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 569/1142] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Jul 20 22:27:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:27:49.746666 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc88c555-28d4-4d51-8ff5-c7cf05bc231b None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:49.841525 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:49.844234 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc88c555-28d4-4d51-8ff5-c7cf05bc231b None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:49.876641 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405d15a7-8fbf-44ea-97ce-86d5c1d2c835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:49.877512 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405d15a7-8fbf-44ea-97ce-86d5c1d2c835 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:49.877797 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405d15a7-8fbf-44ea-97ce-86d5c1d2c835 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:49.878073 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405d15a7-8fbf-44ea-97ce-86d5c1d2c835 None admin] PATH_INFO: `/v3/users/f548c960951b4c75b493e316a2d69135` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:49.891535 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405d15a7-8fbf-44ea-97ce-86d5c1d2c835 None admin] RBAC: Authorizing `identity:get_user(user_id=f548c960951b4c75b493e316a2d69135)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:49.892980 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405d15a7-8fbf-44ea-97ce-86d5c1d2c835 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:49.894600 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 574/1143] 10.222.0.81 () {56 vars in 1177 bytes} [Wed Jul 20 22:27:49 2022] GET /identity/v3/users/f548c960951b4c75b493e316a2d69135 => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:27:49.900394 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bbe492-432e-43af-8284-03efb14577d3 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:49.999041 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bbe492-432e-43af-8284-03efb14577d3 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:50.031193 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1506c862-26e3-4d9c-ac36-68e882bb82a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:50.032099 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1506c862-26e3-4d9c-ac36-68e882bb82a3 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:50.032388 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1506c862-26e3-4d9c-ac36-68e882bb82a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:50.032668 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1506c862-26e3-4d9c-ac36-68e882bb82a3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:50.093017 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:50.099368 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1506c862-26e3-4d9c-ac36-68e882bb82a3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:50.100812 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1506c862-26e3-4d9c-ac36-68e882bb82a3 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:50.198280 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 570/1144] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Jul 20 22:27:49 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 22:27:50.204831 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405d15a7-8fbf-44ea-97ce-86d5c1d2c835 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:50.301219 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405d15a7-8fbf-44ea-97ce-86d5c1d2c835 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:50.333527 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3964d8-16bc-4983-b585-b27612ca113f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:50.334388 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3964d8-16bc-4983-b585-b27612ca113f None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:50.334677 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3964d8-16bc-4983-b585-b27612ca113f None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:50.334957 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3964d8-16bc-4983-b585-b27612ca113f None admin] PATH_INFO: `/v3/projects/d45e33adc8db4d449b67b27692ff2ccd` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:50.336147 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3964d8-16bc-4983-b585-b27612ca113f None admin] RBAC: Authorizing `identity:get_project(project_id=d45e33adc8db4d449b67b27692ff2ccd)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:50.337566 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3964d8-16bc-4983-b585-b27612ca113f None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:50.339262 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 575/1145] 10.222.0.81 () {56 vars in 1186 bytes} [Wed Jul 20 22:27:50 2022] GET /identity/v3/projects/d45e33adc8db4d449b67b27692ff2ccd => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:27:50.346059 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1506c862-26e3-4d9c-ac36-68e882bb82a3 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:50.437745 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:50.443326 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1506c862-26e3-4d9c-ac36-68e882bb82a3 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:50.482362 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb506ab-e703-4465-9373-c05b5708d1dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:50.483043 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb506ab-e703-4465-9373-c05b5708d1dc None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:50.483334 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb506ab-e703-4465-9373-c05b5708d1dc None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:50.483612 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb506ab-e703-4465-9373-c05b5708d1dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:50.484442 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb506ab-e703-4465-9373-c05b5708d1dc None admin] RBAC: Adding query filter params (role.id=d89b6c92d35a401a8517f1d47f6be843, scope.project.id=d45e33adc8db4d449b67b27692ff2ccd, user.id=f548c960951b4c75b493e316a2d69135) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:27:50.484757 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb506ab-e703-4465-9373-c05b5708d1dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:50.486188 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb506ab-e703-4465-9373-c05b5708d1dc None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:50.491711 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 571/1146] 10.222.0.81 () {56 vars in 1374 bytes} [Wed Jul 20 22:27:50 2022] GET /identity/v3/role_assignments?user.id=f548c960951b4c75b493e316a2d69135&scope.project.id=d45e33adc8db4d449b67b27692ff2ccd&role.id=d89b6c92d35a401a8517f1d47f6be843 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:27:52.494019 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817b1ca3-be7b-43e9-875e-721989109f58 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:52.494534 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817b1ca3-be7b-43e9-875e-721989109f58 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:52.494534 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817b1ca3-be7b-43e9-875e-721989109f58 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:52.495878 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 576/1147] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:27:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:27:52.501090 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07cc342-dfb2-47f9-b4b4-b482760e7103 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:52.501485 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07cc342-dfb2-47f9-b4b4-b482760e7103 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:52.501645 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07cc342-dfb2-47f9-b4b4-b482760e7103 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:52.518600 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-e07cc342-dfb2-47f9-b4b4-b482760e7103 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:27:52.519820 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-e07cc342-dfb2-47f9-b4b4-b482760e7103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:27:52.584131 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 572/1148] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:27:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 22:27:52.607081 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817b1ca3-be7b-43e9-875e-721989109f58 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:52.634676 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:52.709524 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817b1ca3-be7b-43e9-875e-721989109f58 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:52.745054 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085f112a-6900-4327-b6fe-78dacf3624e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:52.745792 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085f112a-6900-4327-b6fe-78dacf3624e4 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:52.746081 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085f112a-6900-4327-b6fe-78dacf3624e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:52.746362 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085f112a-6900-4327-b6fe-78dacf3624e4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:52.756421 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085f112a-6900-4327-b6fe-78dacf3624e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 22:27:52.764509 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-085f112a-6900-4327-b6fe-78dacf3624e4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 22:27:52.764509 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:27:52.764509 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:27:52.764509 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:27:52.764509 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:27:52.764509 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:27:52.764509 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:27:52.764509 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:27:52.764509 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:27:52.764509 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:27:52.764509 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:27:52.764509 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:27:52.764509 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 22:27:52.764509 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 22:27:52.764509 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 22:27:52.764509 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 22:27:52.765631 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 22:27:52.765631 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 22:27:52.765631 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 22:27:52.765631 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 22:27:52.765631 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:27:52.765631 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:27:52.765631 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:27:52.765631 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:27:52.765631 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:27:52.765631 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:27:52.765631 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:27:52.765631 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:27:52.765631 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:27:52.765631 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:27:52.765631 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:27:52.765631 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:27:52.766507 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:27:52.766507 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:27:52.766507 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:27:52.766507 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:27:52.766507 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 22:27:52.766507 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 22:27:52.766507 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 22:27:52.766507 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 22:27:52.766507 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 22:27:52.766507 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 22:27:52.766507 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 22:27:52.766507 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:27:52.766507 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 577/1149] 10.222.0.81 () {56 vars in 1096 bytes} [Wed Jul 20 22:27:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:27:52.771994 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07cc342-dfb2-47f9-b4b4-b482760e7103 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:52.863951 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:52.873090 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07cc342-dfb2-47f9-b4b4-b482760e7103 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:52.908169 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da15583b-eee6-4824-8725-cd8402fd32bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:52.908927 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da15583b-eee6-4824-8725-cd8402fd32bd None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:52.909222 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da15583b-eee6-4824-8725-cd8402fd32bd None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:52.909523 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da15583b-eee6-4824-8725-cd8402fd32bd None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:52.910355 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da15583b-eee6-4824-8725-cd8402fd32bd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:27:52.910682 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da15583b-eee6-4824-8725-cd8402fd32bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:52.912112 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da15583b-eee6-4824-8725-cd8402fd32bd None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:52.921304 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 573/1150] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Jul 20 22:27:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:27:52.927467 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085f112a-6900-4327-b6fe-78dacf3624e4 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:53.009595 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:53.024146 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085f112a-6900-4327-b6fe-78dacf3624e4 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:53.057905 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46543c28-4929-44d2-89d7-f081089b23f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:53.058808 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46543c28-4929-44d2-89d7-f081089b23f1 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:53.059118 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46543c28-4929-44d2-89d7-f081089b23f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:53.059397 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46543c28-4929-44d2-89d7-f081089b23f1 None admin] PATH_INFO: `/v3/users/f548c960951b4c75b493e316a2d69135` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:53.060559 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46543c28-4929-44d2-89d7-f081089b23f1 None admin] RBAC: Authorizing `identity:get_user(user_id=f548c960951b4c75b493e316a2d69135)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:53.061974 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46543c28-4929-44d2-89d7-f081089b23f1 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:53.063604 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 578/1151] 10.222.0.81 () {56 vars in 1177 bytes} [Wed Jul 20 22:27:52 2022] GET /identity/v3/users/f548c960951b4c75b493e316a2d69135 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:27:53.069827 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da15583b-eee6-4824-8725-cd8402fd32bd None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:53.165743 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da15583b-eee6-4824-8725-cd8402fd32bd None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:53.197734 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178ef9ce-1fc5-4ff1-91c2-fb6391c3049c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:53.199199 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178ef9ce-1fc5-4ff1-91c2-fb6391c3049c None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:53.199489 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178ef9ce-1fc5-4ff1-91c2-fb6391c3049c None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:53.199763 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178ef9ce-1fc5-4ff1-91c2-fb6391c3049c None admin] PATH_INFO: `/v3/projects/d45e33adc8db4d449b67b27692ff2ccd` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:53.200938 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178ef9ce-1fc5-4ff1-91c2-fb6391c3049c None admin] RBAC: Authorizing `identity:get_project(project_id=d45e33adc8db4d449b67b27692ff2ccd)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:53.202359 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178ef9ce-1fc5-4ff1-91c2-fb6391c3049c None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:53.204084 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 574/1152] 10.222.0.81 () {56 vars in 1186 bytes} [Wed Jul 20 22:27:53 2022] GET /identity/v3/projects/d45e33adc8db4d449b67b27692ff2ccd => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:27:53.210263 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46543c28-4929-44d2-89d7-f081089b23f1 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:53.306173 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46543c28-4929-44d2-89d7-f081089b23f1 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:53.338993 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791994b1-0c74-4868-943f-0b3ba3886971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:53.340461 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791994b1-0c74-4868-943f-0b3ba3886971 None admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:53.340745 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791994b1-0c74-4868-943f-0b3ba3886971 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:53.341024 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791994b1-0c74-4868-943f-0b3ba3886971 None admin] PATH_INFO: `/v3/projects/d45e33adc8db4d449b67b27692ff2ccd/users/f548c960951b4c75b493e316a2d69135/roles/d89b6c92d35a401a8517f1d47f6be843` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:53.342967 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791994b1-0c74-4868-943f-0b3ba3886971 None admin] RBAC: Authorizing `identity:create_grant(project_id=d45e33adc8db4d449b67b27692ff2ccd, user_id=f548c960951b4c75b493e316a2d69135, role_id=d89b6c92d35a401a8517f1d47f6be843)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:53.344421 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791994b1-0c74-4868-943f-0b3ba3886971 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:53.454012 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 579/1153] 10.222.0.81 () {58 vars in 1439 bytes} [Wed Jul 20 22:27:53 2022] PUT /identity/v3/projects/d45e33adc8db4d449b67b27692ff2ccd/users/f548c960951b4c75b493e316a2d69135/roles/d89b6c92d35a401a8517f1d47f6be843 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:27:55.442174 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f666a920-972d-40c4-9009-55fc9767391b None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:55.442732 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f666a920-972d-40c4-9009-55fc9767391b None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:55.442732 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f666a920-972d-40c4-9009-55fc9767391b None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:55.444201 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 575/1154] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:27:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:27:55.449738 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec927ea-dc0b-4b2e-a955-4fe8ff80934b None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:55.450121 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec927ea-dc0b-4b2e-a955-4fe8ff80934b None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:55.450294 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec927ea-dc0b-4b2e-a955-4fe8ff80934b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:55.470326 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-6ec927ea-dc0b-4b2e-a955-4fe8ff80934b None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:27:55.471616 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-6ec927ea-dc0b-4b2e-a955-4fe8ff80934b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:27:55.480727 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:55.537684 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 580/1155] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:27:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 22:27:55.560837 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f666a920-972d-40c4-9009-55fc9767391b None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:55.582900 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:55.661237 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f666a920-972d-40c4-9009-55fc9767391b None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:55.693555 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcb8ceb-ec2f-4032-84f7-c35b8a0c5719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:55.694403 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcb8ceb-ec2f-4032-84f7-c35b8a0c5719 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:55.694635 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcb8ceb-ec2f-4032-84f7-c35b8a0c5719 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:55.694912 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcb8ceb-ec2f-4032-84f7-c35b8a0c5719 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:55.705640 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcb8ceb-ec2f-4032-84f7-c35b8a0c5719 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 22:27:55.713640 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-fbcb8ceb-ec2f-4032-84f7-c35b8a0c5719 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 22:27:55.713640 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:27:55.713640 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:27:55.713640 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:27:55.713640 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:27:55.713640 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:27:55.713640 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:27:55.713640 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:27:55.713640 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:27:55.713640 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:27:55.713640 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:27:55.713640 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:27:55.713640 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 22:27:55.713640 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 22:27:55.713640 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 22:27:55.713640 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 22:27:55.715295 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 22:27:55.715295 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 22:27:55.715295 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 22:27:55.715295 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 22:27:55.715295 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:27:55.715295 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:27:55.715295 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:27:55.715295 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:27:55.715295 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:27:55.715295 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:27:55.715295 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:27:55.715295 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:27:55.715295 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:27:55.715295 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:27:55.715295 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:27:55.715295 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:27:55.716467 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:27:55.716467 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:27:55.716467 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:27:55.716467 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:27:55.716467 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 22:27:55.716467 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 22:27:55.716467 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 22:27:55.716467 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 22:27:55.716467 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 22:27:55.716467 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 22:27:55.716467 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 22:27:55.716467 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:27:55.716467 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 576/1156] 10.222.0.81 () {56 vars in 1096 bytes} [Wed Jul 20 22:27:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:27:55.721302 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec927ea-dc0b-4b2e-a955-4fe8ff80934b None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:55.844819 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec927ea-dc0b-4b2e-a955-4fe8ff80934b None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:55.878154 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80fd1df-b9f1-48af-a7aa-0301612e6fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:55.878820 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80fd1df-b9f1-48af-a7aa-0301612e6fff None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:55.879138 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80fd1df-b9f1-48af-a7aa-0301612e6fff None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:55.879427 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80fd1df-b9f1-48af-a7aa-0301612e6fff None admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:55.880171 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80fd1df-b9f1-48af-a7aa-0301612e6fff None admin] RBAC: Adding query filter params (name=admin) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:27:55.880483 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80fd1df-b9f1-48af-a7aa-0301612e6fff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:55.881850 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80fd1df-b9f1-48af-a7aa-0301612e6fff None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:55.890389 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 581/1157] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Jul 20 22:27:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:27:55.896179 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcb8ceb-ec2f-4032-84f7-c35b8a0c5719 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:55.973726 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:55.992153 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcb8ceb-ec2f-4032-84f7-c35b8a0c5719 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:56.025073 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59be665-a57c-4d8c-adf4-e5edf0404b91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:56.025955 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59be665-a57c-4d8c-adf4-e5edf0404b91 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:56.026242 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59be665-a57c-4d8c-adf4-e5edf0404b91 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:56.026559 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59be665-a57c-4d8c-adf4-e5edf0404b91 None admin] PATH_INFO: `/v3/users/f548c960951b4c75b493e316a2d69135` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:56.027765 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59be665-a57c-4d8c-adf4-e5edf0404b91 None admin] RBAC: Authorizing `identity:get_user(user_id=f548c960951b4c75b493e316a2d69135)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:56.029188 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59be665-a57c-4d8c-adf4-e5edf0404b91 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:56.030802 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 577/1158] 10.222.0.81 () {56 vars in 1177 bytes} [Wed Jul 20 22:27:55 2022] GET /identity/v3/users/f548c960951b4c75b493e316a2d69135 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:27:56.036721 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80fd1df-b9f1-48af-a7aa-0301612e6fff None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:56.046374 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:56.137224 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80fd1df-b9f1-48af-a7aa-0301612e6fff None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:56.170782 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d597a7-0eb4-4666-8516-69584c1209d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:56.171457 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d597a7-0eb4-4666-8516-69584c1209d9 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:56.171749 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d597a7-0eb4-4666-8516-69584c1209d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:56.172033 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d597a7-0eb4-4666-8516-69584c1209d9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:56.237349 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d597a7-0eb4-4666-8516-69584c1209d9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:56.238736 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d597a7-0eb4-4666-8516-69584c1209d9 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:56.246839 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:56.334540 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 582/1159] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Jul 20 22:27:56 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 22:27:56.340994 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59be665-a57c-4d8c-adf4-e5edf0404b91 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:56.437028 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59be665-a57c-4d8c-adf4-e5edf0404b91 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:56.472230 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebab9b4-5f46-425e-bd27-5fe3c1b9df8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:56.473113 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebab9b4-5f46-425e-bd27-5fe3c1b9df8c None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:56.473400 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebab9b4-5f46-425e-bd27-5fe3c1b9df8c None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:56.473682 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebab9b4-5f46-425e-bd27-5fe3c1b9df8c None admin] PATH_INFO: `/v3/projects/d45e33adc8db4d449b67b27692ff2ccd` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:56.474860 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebab9b4-5f46-425e-bd27-5fe3c1b9df8c None admin] RBAC: Authorizing `identity:get_project(project_id=d45e33adc8db4d449b67b27692ff2ccd)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:56.476324 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebab9b4-5f46-425e-bd27-5fe3c1b9df8c None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:56.478035 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 578/1160] 10.222.0.81 () {56 vars in 1186 bytes} [Wed Jul 20 22:27:56 2022] GET /identity/v3/projects/d45e33adc8db4d449b67b27692ff2ccd => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:27:56.484804 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d597a7-0eb4-4666-8516-69584c1209d9 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:56.582717 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d597a7-0eb4-4666-8516-69584c1209d9 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:56.590418 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:56.616616 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a77432-e84e-4675-b9cd-5929e57746e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:56.617266 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a77432-e84e-4675-b9cd-5929e57746e6 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:56.617555 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a77432-e84e-4675-b9cd-5929e57746e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:56.617839 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a77432-e84e-4675-b9cd-5929e57746e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:56.618662 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a77432-e84e-4675-b9cd-5929e57746e6 None admin] RBAC: Adding query filter params (role.id=d89b6c92d35a401a8517f1d47f6be843, scope.project.id=d45e33adc8db4d449b67b27692ff2ccd, user.id=f548c960951b4c75b493e316a2d69135) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:27:56.618979 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a77432-e84e-4675-b9cd-5929e57746e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:56.620458 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a77432-e84e-4675-b9cd-5929e57746e6 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:56.626204 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 583/1161] 10.222.0.81 () {56 vars in 1374 bytes} [Wed Jul 20 22:27:56 2022] GET /identity/v3/role_assignments?user.id=f548c960951b4c75b493e316a2d69135&scope.project.id=d45e33adc8db4d449b67b27692ff2ccd&role.id=d89b6c92d35a401a8517f1d47f6be843 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:27:58.669408 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fc51a3-0b8a-49e3-980c-436fbf6cb250 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:58.669990 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fc51a3-0b8a-49e3-980c-436fbf6cb250 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:58.669990 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fc51a3-0b8a-49e3-980c-436fbf6cb250 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:58.671373 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 579/1162] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:27:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:27:58.676571 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3ee836-a9a3-4212-8024-44a995c8a63a None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:58.677015 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3ee836-a9a3-4212-8024-44a995c8a63a None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:58.677140 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3ee836-a9a3-4212-8024-44a995c8a63a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:58.693886 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-fa3ee836-a9a3-4212-8024-44a995c8a63a None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:27:58.695100 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-fa3ee836-a9a3-4212-8024-44a995c8a63a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:27:58.759026 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 584/1163] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:27:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 22:27:58.782561 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fc51a3-0b8a-49e3-980c-436fbf6cb250 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:58.800877 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:58.881295 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fc51a3-0b8a-49e3-980c-436fbf6cb250 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:58.914183 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37db5c73-35f2-41cb-bc23-7e48987a369b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:58.914820 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37db5c73-35f2-41cb-bc23-7e48987a369b None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:58.915142 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37db5c73-35f2-41cb-bc23-7e48987a369b None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:58.915423 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37db5c73-35f2-41cb-bc23-7e48987a369b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:58.980242 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37db5c73-35f2-41cb-bc23-7e48987a369b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:58.981610 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37db5c73-35f2-41cb-bc23-7e48987a369b None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:58.998216 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:59.076005 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 580/1164] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Jul 20 22:27:58 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 22:27:59.081944 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3ee836-a9a3-4212-8024-44a995c8a63a None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:59.176330 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:59.179285 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3ee836-a9a3-4212-8024-44a995c8a63a None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:59.212667 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067c7c6b-d620-4c34-8b63-869eb2cd39d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:59.213489 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067c7c6b-d620-4c34-8b63-869eb2cd39d6 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:59.213773 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067c7c6b-d620-4c34-8b63-869eb2cd39d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:59.214051 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067c7c6b-d620-4c34-8b63-869eb2cd39d6 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:59.220513 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067c7c6b-d620-4c34-8b63-869eb2cd39d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:59.221930 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067c7c6b-d620-4c34-8b63-869eb2cd39d6 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:59.237367 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-067c7c6b-d620-4c34-8b63-869eb2cd39d6 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 20 22:27:59.237367 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:27:59.237367 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 20 22:27:59.237367 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 20 22:27:59.237367 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 22:27:59.237367 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 22:27:59.237367 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 22:27:59.237367 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 22:27:59.237367 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 20 22:27:59.237367 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:27:59.237367 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 22:27:59.237367 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:27:59.237367 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:27:59.237367 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:27:59.237367 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:27:59.237367 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:27:59.237367 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:27:59.238599 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:27:59.238599 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:27:59.238599 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:27:59.238599 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:27:59.238599 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:27:59.238599 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:27:59.238599 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 22:27:59.238599 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 22:27:59.238599 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 22:27:59.238599 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 22:27:59.238599 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:27:59.238599 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:27:59.238599 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:27:59.238599 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:27:59.238599 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:27:59.238599 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:27:59.239754 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:27:59.239754 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:27:59.239754 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:27:59.239754 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:27:59.239754 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:27:59.239754 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:27:59.239754 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:27:59.239754 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:27:59.239754 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:27:59.239754 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:27:59.239754 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 20 22:27:59.239754 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 22:27:59.239754 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 20 22:27:59.239754 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:27:59.239754 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 585/1165] 10.222.0.81 () {56 vars in 1117 bytes} [Wed Jul 20 22:27:59 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:27:59.244487 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37db5c73-35f2-41cb-bc23-7e48987a369b None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:27:59.339848 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37db5c73-35f2-41cb-bc23-7e48987a369b None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:27:59.356031 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:27:59.373592 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c8bbbe-ed13-4a85-95d8-8dfbc12697d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:27:59.374254 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c8bbbe-ed13-4a85-95d8-8dfbc12697d4 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:27:59.374541 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c8bbbe-ed13-4a85-95d8-8dfbc12697d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:27:59.374821 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c8bbbe-ed13-4a85-95d8-8dfbc12697d4 None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:27:59.375564 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c8bbbe-ed13-4a85-95d8-8dfbc12697d4 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:27:59.375871 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c8bbbe-ed13-4a85-95d8-8dfbc12697d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:27:59.377216 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c8bbbe-ed13-4a85-95d8-8dfbc12697d4 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:27:59.384136 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 581/1166] 10.222.0.81 () {56 vars in 1115 bytes} [Wed Jul 20 22:27:59 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:28:01.438948 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedabaac-b6d6-4000-9cad-554e211caa26 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:28:01.439509 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedabaac-b6d6-4000-9cad-554e211caa26 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:28:01.439610 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedabaac-b6d6-4000-9cad-554e211caa26 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:28:01.440936 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 586/1167] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:28:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:28:01.446196 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef3292c-f441-4ccb-ab89-5a3732116fc4 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:28:01.446622 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef3292c-f441-4ccb-ab89-5a3732116fc4 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:28:01.446735 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef3292c-f441-4ccb-ab89-5a3732116fc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:28:01.462919 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-0ef3292c-f441-4ccb-ab89-5a3732116fc4 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:28:01.464127 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-0ef3292c-f441-4ccb-ab89-5a3732116fc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:28:01.529840 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 582/1168] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:28:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 22:28:01.552818 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedabaac-b6d6-4000-9cad-554e211caa26 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:28:01.652536 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedabaac-b6d6-4000-9cad-554e211caa26 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:28:01.685949 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190329d8-05a7-42c3-af17-ab0918aa550a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:28:01.686599 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190329d8-05a7-42c3-af17-ab0918aa550a None admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:28:01.686883 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190329d8-05a7-42c3-af17-ab0918aa550a None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:28:01.687195 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190329d8-05a7-42c3-af17-ab0918aa550a None admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:28:01.687943 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190329d8-05a7-42c3-af17-ab0918aa550a None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:28:01.689305 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190329d8-05a7-42c3-af17-ab0918aa550a None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:28:01.789430 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 587/1169] 10.222.0.81 () {60 vars in 1138 bytes} [Wed Jul 20 22:28:01 2022] POST /identity/v3/domains => generated 267 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:28:03.842922 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8886696-f81d-45af-9d5e-961c0a6a866a None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:28:03.843519 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8886696-f81d-45af-9d5e-961c0a6a866a None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:28:03.843519 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8886696-f81d-45af-9d5e-961c0a6a866a None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:28:03.844886 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 583/1170] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:28:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:28:03.850016 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c88b49-627c-4b2f-8c60-484b40c5e545 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:28:03.850420 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c88b49-627c-4b2f-8c60-484b40c5e545 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:28:03.850578 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c88b49-627c-4b2f-8c60-484b40c5e545 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:28:03.868103 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-98c88b49-627c-4b2f-8c60-484b40c5e545 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:28:03.869935 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-98c88b49-627c-4b2f-8c60-484b40c5e545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:28:03.886964 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:28:03.952679 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 588/1171] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:28:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 22:28:03.976174 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8886696-f81d-45af-9d5e-961c0a6a866a None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:28:04.074128 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8886696-f81d-45af-9d5e-961c0a6a866a None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:28:04.084536 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:28:04.107772 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4705f7fc-3394-4bde-8902-9ea7a99ba0ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:28:04.108586 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4705f7fc-3394-4bde-8902-9ea7a99ba0ce None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:28:04.108873 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4705f7fc-3394-4bde-8902-9ea7a99ba0ce None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:28:04.109155 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4705f7fc-3394-4bde-8902-9ea7a99ba0ce None admin] PATH_INFO: `/v3/domains/8e1da46693514ca9b020a6171fdec27d` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:28:04.119374 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4705f7fc-3394-4bde-8902-9ea7a99ba0ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=8e1da46693514ca9b020a6171fdec27d)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:28:04.121319 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4705f7fc-3394-4bde-8902-9ea7a99ba0ce None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:28:04.123283 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 584/1172] 10.222.0.81 () {56 vars in 1183 bytes} [Wed Jul 20 22:28:03 2022] GET /identity/v3/domains/8e1da46693514ca9b020a6171fdec27d => generated 267 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:28:04.128775 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c88b49-627c-4b2f-8c60-484b40c5e545 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:28:04.226197 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c88b49-627c-4b2f-8c60-484b40c5e545 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:28:04.260196 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355303f3-3766-4df7-bda8-8a888a1fa227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:28:04.260849 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355303f3-3766-4df7-bda8-8a888a1fa227 None admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:28:04.261137 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355303f3-3766-4df7-bda8-8a888a1fa227 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:28:04.261519 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355303f3-3766-4df7-bda8-8a888a1fa227 None admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:28:04.262240 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355303f3-3766-4df7-bda8-8a888a1fa227 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:28:04.263699 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355303f3-3766-4df7-bda8-8a888a1fa227 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:28:04.373000 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 589/1173] 10.222.0.81 () {60 vars in 1141 bytes} [Wed Jul 20 22:28:04 2022] POST /identity/v3/projects => generated 361 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:28:06.427108 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d512014c-2255-480c-bcdc-b72b1d70c280 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:28:06.427660 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d512014c-2255-480c-bcdc-b72b1d70c280 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:28:06.427660 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d512014c-2255-480c-bcdc-b72b1d70c280 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:28:06.429029 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 585/1174] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:28:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:28:06.434333 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187fe345-7d6a-400b-91e8-f7ec4b36f798 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:28:06.434743 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187fe345-7d6a-400b-91e8-f7ec4b36f798 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:28:06.434902 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187fe345-7d6a-400b-91e8-f7ec4b36f798 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:28:06.439871 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:28:06.451775 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-187fe345-7d6a-400b-91e8-f7ec4b36f798 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:28:06.452913 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-187fe345-7d6a-400b-91e8-f7ec4b36f798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:28:06.518288 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 590/1175] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:28:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 22:28:06.544617 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d512014c-2255-480c-bcdc-b72b1d70c280 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:28:06.650517 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d512014c-2255-480c-bcdc-b72b1d70c280 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:28:06.683589 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2791dba5-7dd6-454e-9d63-76497b45aa68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:28:06.684465 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2791dba5-7dd6-454e-9d63-76497b45aa68 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:28:06.684750 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2791dba5-7dd6-454e-9d63-76497b45aa68 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:28:06.685026 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2791dba5-7dd6-454e-9d63-76497b45aa68 None admin] PATH_INFO: `/v3/domains/8e1da46693514ca9b020a6171fdec27d` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:28:06.686192 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2791dba5-7dd6-454e-9d63-76497b45aa68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=8e1da46693514ca9b020a6171fdec27d)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:28:06.687642 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2791dba5-7dd6-454e-9d63-76497b45aa68 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:28:06.689257 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 586/1176] 10.222.0.81 () {56 vars in 1183 bytes} [Wed Jul 20 22:28:06 2022] GET /identity/v3/domains/8e1da46693514ca9b020a6171fdec27d => generated 267 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:28:06.695116 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187fe345-7d6a-400b-91e8-f7ec4b36f798 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:28:06.792428 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187fe345-7d6a-400b-91e8-f7ec4b36f798 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:28:06.820624 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:28:06.825329 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6a08a0-b6f1-4e69-bd5d-65da65a13fb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:28:06.825984 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6a08a0-b6f1-4e69-bd5d-65da65a13fb9 None admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:28:06.826266 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6a08a0-b6f1-4e69-bd5d-65da65a13fb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:28:06.826564 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6a08a0-b6f1-4e69-bd5d-65da65a13fb9 None admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:28:06.827286 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6a08a0-b6f1-4e69-bd5d-65da65a13fb9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:28:06.832987 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6a08a0-b6f1-4e69-bd5d-65da65a13fb9 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:28:06.921030 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 591/1177] 10.222.0.81 () {60 vars in 1132 bytes} [Wed Jul 20 22:28:06 2022] POST /identity/v3/users => generated 311 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:28:08.979874 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50591888-a5d1-4670-9a0d-76a0dd5240f4 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:28:08.980405 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50591888-a5d1-4670-9a0d-76a0dd5240f4 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:28:08.980405 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50591888-a5d1-4670-9a0d-76a0dd5240f4 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:28:08.981794 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 587/1178] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:28:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:28:08.986890 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc16fa1b-519e-4013-bc86-be9c0ce19c7c None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:28:08.987351 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc16fa1b-519e-4013-bc86-be9c0ce19c7c None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:28:08.987602 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc16fa1b-519e-4013-bc86-be9c0ce19c7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:28:09.004026 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-dc16fa1b-519e-4013-bc86-be9c0ce19c7c None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:28:09.005217 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-dc16fa1b-519e-4013-bc86-be9c0ce19c7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:28:09.070251 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 592/1179] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:28:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 22:28:09.094536 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50591888-a5d1-4670-9a0d-76a0dd5240f4 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:28:09.134253 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:28:09.193580 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50591888-a5d1-4670-9a0d-76a0dd5240f4 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:28:09.225890 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166be51d-c672-4fd7-8074-0fbc66d99c2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:28:09.226693 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166be51d-c672-4fd7-8074-0fbc66d99c2f None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:28:09.226978 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166be51d-c672-4fd7-8074-0fbc66d99c2f None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:28:09.227296 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166be51d-c672-4fd7-8074-0fbc66d99c2f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:28:09.238237 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166be51d-c672-4fd7-8074-0fbc66d99c2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 22:28:09.246893 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-166be51d-c672-4fd7-8074-0fbc66d99c2f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 22:28:09.246893 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:28:09.246893 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:28:09.246893 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:28:09.246893 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:28:09.246893 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:28:09.246893 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:28:09.246893 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:28:09.246893 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:28:09.246893 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:28:09.246893 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:28:09.246893 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:28:09.246893 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 22:28:09.246893 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 22:28:09.246893 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 22:28:09.246893 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 22:28:09.248039 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 22:28:09.248039 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 22:28:09.248039 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 22:28:09.248039 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 22:28:09.248039 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:28:09.248039 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:28:09.248039 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:28:09.248039 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:28:09.248039 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:28:09.248039 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:28:09.248039 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:28:09.248039 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:28:09.248039 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:28:09.248039 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:28:09.248039 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:28:09.248039 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:28:09.249086 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:28:09.249086 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:28:09.249086 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:28:09.249086 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:28:09.249086 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 22:28:09.249086 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 22:28:09.249086 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 22:28:09.249086 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 22:28:09.249086 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 22:28:09.249086 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 22:28:09.249086 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 22:28:09.249086 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:28:09.249086 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 588/1180] 10.222.0.81 () {56 vars in 1096 bytes} [Wed Jul 20 22:28:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:28:09.254068 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc16fa1b-519e-4013-bc86-be9c0ce19c7c None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:28:09.350399 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc16fa1b-519e-4013-bc86-be9c0ce19c7c None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:28:09.367470 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:28:09.383668 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1184bbc1-1193-4446-b0e2-63ecb657a5ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:28:09.384408 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1184bbc1-1193-4446-b0e2-63ecb657a5ef None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:28:09.384699 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1184bbc1-1193-4446-b0e2-63ecb657a5ef None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:28:09.384984 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1184bbc1-1193-4446-b0e2-63ecb657a5ef None admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:28:09.385728 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1184bbc1-1193-4446-b0e2-63ecb657a5ef None admin] RBAC: Adding query filter params (name=admin) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:28:09.386041 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1184bbc1-1193-4446-b0e2-63ecb657a5ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:28:09.387493 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1184bbc1-1193-4446-b0e2-63ecb657a5ef None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:28:09.395278 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 593/1181] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Jul 20 22:28:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:28:09.401483 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166be51d-c672-4fd7-8074-0fbc66d99c2f None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:28:09.494705 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:28:09.498535 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166be51d-c672-4fd7-8074-0fbc66d99c2f None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:28:09.532447 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3fbecd-b3e3-4b7f-a06e-2d3dc85b581e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:28:09.533332 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3fbecd-b3e3-4b7f-a06e-2d3dc85b581e None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:28:09.533620 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3fbecd-b3e3-4b7f-a06e-2d3dc85b581e None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:28:09.533901 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3fbecd-b3e3-4b7f-a06e-2d3dc85b581e None admin] PATH_INFO: `/v3/users/fdad1f3206ea49d8bcabd5a8403b2cd6` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:28:09.546373 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3fbecd-b3e3-4b7f-a06e-2d3dc85b581e None admin] RBAC: Authorizing `identity:get_user(user_id=fdad1f3206ea49d8bcabd5a8403b2cd6)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:28:09.547924 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3fbecd-b3e3-4b7f-a06e-2d3dc85b581e None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:28:09.549901 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 589/1182] 10.222.0.81 () {56 vars in 1177 bytes} [Wed Jul 20 22:28:09 2022] GET /identity/v3/users/fdad1f3206ea49d8bcabd5a8403b2cd6 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 22:28:09.555465 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1184bbc1-1193-4446-b0e2-63ecb657a5ef None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:28:09.652962 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1184bbc1-1193-4446-b0e2-63ecb657a5ef None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:28:09.686179 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f090189-41c7-4785-9cb5-c0533da87983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:28:09.686824 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f090189-41c7-4785-9cb5-c0533da87983 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:28:09.687136 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f090189-41c7-4785-9cb5-c0533da87983 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:28:09.687417 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f090189-41c7-4785-9cb5-c0533da87983 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:28:09.733151 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:28:09.752489 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f090189-41c7-4785-9cb5-c0533da87983 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:28:09.753876 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f090189-41c7-4785-9cb5-c0533da87983 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:28:09.853842 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 594/1183] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Jul 20 22:28:09 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 22:28:09.859723 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3fbecd-b3e3-4b7f-a06e-2d3dc85b581e None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:28:09.960806 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3fbecd-b3e3-4b7f-a06e-2d3dc85b581e None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:28:09.993402 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79dd09c-1c03-4098-a86a-6e97bda11a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:28:09.994780 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79dd09c-1c03-4098-a86a-6e97bda11a33 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:28:09.995081 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79dd09c-1c03-4098-a86a-6e97bda11a33 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:28:09.995355 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79dd09c-1c03-4098-a86a-6e97bda11a33 None admin] PATH_INFO: `/v3/projects/35a4b0462b7f471eba8a159a67bc9d7e` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:28:09.996541 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79dd09c-1c03-4098-a86a-6e97bda11a33 None admin] RBAC: Authorizing `identity:get_project(project_id=35a4b0462b7f471eba8a159a67bc9d7e)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:28:09.997949 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79dd09c-1c03-4098-a86a-6e97bda11a33 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:28:09.999753 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 590/1184] 10.222.0.81 () {56 vars in 1186 bytes} [Wed Jul 20 22:28:09 2022] GET /identity/v3/projects/35a4b0462b7f471eba8a159a67bc9d7e => generated 361 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:28:10.015541 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f090189-41c7-4785-9cb5-c0533da87983 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:28:10.084102 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:28:10.102006 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f090189-41c7-4785-9cb5-c0533da87983 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:28:10.134975 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43203671-70fa-4abd-b77c-9dfdf593da3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:28:10.135699 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43203671-70fa-4abd-b77c-9dfdf593da3a None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:28:10.135980 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43203671-70fa-4abd-b77c-9dfdf593da3a None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:28:10.136283 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43203671-70fa-4abd-b77c-9dfdf593da3a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:28:10.137122 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43203671-70fa-4abd-b77c-9dfdf593da3a None admin] RBAC: Adding query filter params (role.id=d89b6c92d35a401a8517f1d47f6be843, scope.project.id=35a4b0462b7f471eba8a159a67bc9d7e, user.id=fdad1f3206ea49d8bcabd5a8403b2cd6) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:28:10.137430 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43203671-70fa-4abd-b77c-9dfdf593da3a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:28:10.138906 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43203671-70fa-4abd-b77c-9dfdf593da3a None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:28:10.144224 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 595/1185] 10.222.0.81 () {56 vars in 1374 bytes} [Wed Jul 20 22:28:10 2022] GET /identity/v3/role_assignments?user.id=fdad1f3206ea49d8bcabd5a8403b2cd6&scope.project.id=35a4b0462b7f471eba8a159a67bc9d7e&role.id=d89b6c92d35a401a8517f1d47f6be843 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:28:12.141103 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e4ce89-42b1-4e85-b09c-a144c86ae755 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:28:12.141659 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e4ce89-42b1-4e85-b09c-a144c86ae755 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:28:12.141774 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e4ce89-42b1-4e85-b09c-a144c86ae755 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:28:12.143138 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 591/1186] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:28:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:28:12.148173 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7506cb9-9331-4afb-812f-1861324e7607 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:28:12.148603 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7506cb9-9331-4afb-812f-1861324e7607 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:28:12.148732 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7506cb9-9331-4afb-812f-1861324e7607 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:28:12.166555 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-d7506cb9-9331-4afb-812f-1861324e7607 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:28:12.167757 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-d7506cb9-9331-4afb-812f-1861324e7607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:28:12.236571 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 596/1187] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:28:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 22:28:12.260464 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e4ce89-42b1-4e85-b09c-a144c86ae755 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:28:12.285810 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:28:12.366381 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e4ce89-42b1-4e85-b09c-a144c86ae755 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:28:12.399455 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8188212-9d37-4ce9-be37-a42c98bcc5bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:28:12.400225 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8188212-9d37-4ce9-be37-a42c98bcc5bb None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:28:12.400512 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8188212-9d37-4ce9-be37-a42c98bcc5bb None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:28:12.400789 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8188212-9d37-4ce9-be37-a42c98bcc5bb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:28:12.410711 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8188212-9d37-4ce9-be37-a42c98bcc5bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 22:28:12.418842 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-c8188212-9d37-4ce9-be37-a42c98bcc5bb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 22:28:12.418842 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:28:12.418842 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:28:12.418842 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:28:12.418842 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:28:12.418842 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:28:12.418842 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:28:12.418842 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:28:12.418842 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:28:12.418842 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:28:12.418842 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:28:12.418842 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:28:12.418842 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 22:28:12.418842 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 22:28:12.418842 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 22:28:12.418842 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 22:28:12.420040 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 22:28:12.420040 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 22:28:12.420040 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 22:28:12.420040 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 22:28:12.420040 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:28:12.420040 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:28:12.420040 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:28:12.420040 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:28:12.420040 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:28:12.420040 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:28:12.420040 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:28:12.420040 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:28:12.420040 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:28:12.420040 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:28:12.420040 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:28:12.420040 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:28:12.421071 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:28:12.421071 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:28:12.421071 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:28:12.421071 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:28:12.421071 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 22:28:12.421071 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 22:28:12.421071 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 22:28:12.421071 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 22:28:12.421071 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 22:28:12.421071 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 22:28:12.421071 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 22:28:12.421071 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:28:12.421071 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 592/1188] 10.222.0.81 () {56 vars in 1096 bytes} [Wed Jul 20 22:28:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:28:12.426179 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7506cb9-9331-4afb-812f-1861324e7607 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:28:12.498445 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:28:12.522020 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7506cb9-9331-4afb-812f-1861324e7607 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:28:12.556303 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1fa41e-3bcc-4497-af29-a18ccc254828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:28:12.557053 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1fa41e-3bcc-4497-af29-a18ccc254828 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:28:12.557337 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1fa41e-3bcc-4497-af29-a18ccc254828 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:28:12.557612 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1fa41e-3bcc-4497-af29-a18ccc254828 None admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:28:12.558670 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1fa41e-3bcc-4497-af29-a18ccc254828 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:28:12.558992 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1fa41e-3bcc-4497-af29-a18ccc254828 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:28:12.560392 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1fa41e-3bcc-4497-af29-a18ccc254828 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:28:12.568402 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 597/1189] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Jul 20 22:28:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:28:12.574397 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8188212-9d37-4ce9-be37-a42c98bcc5bb None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:28:12.659644 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:28:12.675133 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8188212-9d37-4ce9-be37-a42c98bcc5bb None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:28:12.708529 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f139d4-7bb9-4854-a839-0856b3e256a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:28:12.709415 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f139d4-7bb9-4854-a839-0856b3e256a7 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:28:12.709702 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f139d4-7bb9-4854-a839-0856b3e256a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:28:12.709984 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f139d4-7bb9-4854-a839-0856b3e256a7 None admin] PATH_INFO: `/v3/users/fdad1f3206ea49d8bcabd5a8403b2cd6` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:28:12.711198 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f139d4-7bb9-4854-a839-0856b3e256a7 None admin] RBAC: Authorizing `identity:get_user(user_id=fdad1f3206ea49d8bcabd5a8403b2cd6)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:28:12.712654 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f139d4-7bb9-4854-a839-0856b3e256a7 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:28:12.714283 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 593/1190] 10.222.0.81 () {56 vars in 1177 bytes} [Wed Jul 20 22:28:12 2022] GET /identity/v3/users/fdad1f3206ea49d8bcabd5a8403b2cd6 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:28:12.720497 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1fa41e-3bcc-4497-af29-a18ccc254828 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:28:12.818755 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1fa41e-3bcc-4497-af29-a18ccc254828 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:28:12.851222 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b609615-9070-488f-9694-28fccc9a25ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:28:12.852094 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b609615-9070-488f-9694-28fccc9a25ce None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:28:12.852381 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b609615-9070-488f-9694-28fccc9a25ce None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:28:12.853168 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b609615-9070-488f-9694-28fccc9a25ce None admin] PATH_INFO: `/v3/projects/35a4b0462b7f471eba8a159a67bc9d7e` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:28:12.854344 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b609615-9070-488f-9694-28fccc9a25ce None admin] RBAC: Authorizing `identity:get_project(project_id=35a4b0462b7f471eba8a159a67bc9d7e)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:28:12.855802 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b609615-9070-488f-9694-28fccc9a25ce None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:28:12.857487 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 598/1191] 10.222.0.81 () {56 vars in 1186 bytes} [Wed Jul 20 22:28:12 2022] GET /identity/v3/projects/35a4b0462b7f471eba8a159a67bc9d7e => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:28:12.863495 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f139d4-7bb9-4854-a839-0856b3e256a7 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:28:12.959971 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f139d4-7bb9-4854-a839-0856b3e256a7 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:28:12.995422 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1fe643-2f7f-4e2e-bd76-03e78c5979bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:28:12.996339 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1fe643-2f7f-4e2e-bd76-03e78c5979bb None admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:28:12.996630 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1fe643-2f7f-4e2e-bd76-03e78c5979bb None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:28:12.996910 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1fe643-2f7f-4e2e-bd76-03e78c5979bb None admin] PATH_INFO: `/v3/projects/35a4b0462b7f471eba8a159a67bc9d7e/users/fdad1f3206ea49d8bcabd5a8403b2cd6/roles/d89b6c92d35a401a8517f1d47f6be843` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:28:12.998917 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1fe643-2f7f-4e2e-bd76-03e78c5979bb None admin] RBAC: Authorizing `identity:create_grant(project_id=35a4b0462b7f471eba8a159a67bc9d7e, user_id=fdad1f3206ea49d8bcabd5a8403b2cd6, role_id=d89b6c92d35a401a8517f1d47f6be843)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:28:13.000503 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1fe643-2f7f-4e2e-bd76-03e78c5979bb None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:28:13.136698 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 594/1192] 10.222.0.81 () {58 vars in 1439 bytes} [Wed Jul 20 22:28:12 2022] PUT /identity/v3/projects/35a4b0462b7f471eba8a159a67bc9d7e/users/fdad1f3206ea49d8bcabd5a8403b2cd6/roles/d89b6c92d35a401a8517f1d47f6be843 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:28:15.149111 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1844641-6422-479f-a2c7-d8910c4ded8c None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:28:15.149672 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1844641-6422-479f-a2c7-d8910c4ded8c None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:28:15.149672 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1844641-6422-479f-a2c7-d8910c4ded8c None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:28:15.151051 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 599/1193] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:28:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:28:15.156757 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1533711-7f60-4a12-8ff5-b63c8aea2080 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:28:15.157194 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1533711-7f60-4a12-8ff5-b63c8aea2080 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:28:15.157328 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1533711-7f60-4a12-8ff5-b63c8aea2080 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:28:15.174386 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-a1533711-7f60-4a12-8ff5-b63c8aea2080 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:28:15.175609 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-a1533711-7f60-4a12-8ff5-b63c8aea2080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:28:15.183438 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:28:15.240566 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 595/1194] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:28:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 22:28:15.265488 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1844641-6422-479f-a2c7-d8910c4ded8c None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:28:15.269350 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:28:15.366402 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1844641-6422-479f-a2c7-d8910c4ded8c None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:28:15.400702 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0317f08e-0301-405e-9e0b-7e78388e1f3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:28:15.401425 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0317f08e-0301-405e-9e0b-7e78388e1f3f None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:28:15.401704 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0317f08e-0301-405e-9e0b-7e78388e1f3f None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:28:15.401979 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0317f08e-0301-405e-9e0b-7e78388e1f3f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:28:15.412112 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0317f08e-0301-405e-9e0b-7e78388e1f3f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 22:28:15.420834 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-0317f08e-0301-405e-9e0b-7e78388e1f3f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 22:28:15.420834 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:28:15.420834 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:28:15.420834 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:28:15.420834 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:28:15.420834 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:28:15.420834 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:28:15.420834 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:28:15.420834 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:28:15.420834 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:28:15.420834 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:28:15.420834 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:28:15.420834 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 22:28:15.420834 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 22:28:15.420834 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 22:28:15.420834 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 22:28:15.421982 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 22:28:15.421982 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 22:28:15.421982 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 22:28:15.421982 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 22:28:15.421982 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:28:15.421982 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:28:15.421982 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:28:15.421982 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:28:15.421982 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:28:15.421982 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:28:15.421982 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:28:15.421982 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:28:15.421982 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:28:15.421982 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:28:15.421982 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:28:15.421982 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:28:15.423097 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:28:15.423097 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:28:15.423097 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:28:15.423097 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:28:15.423097 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 22:28:15.423097 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 22:28:15.423097 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 22:28:15.423097 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 22:28:15.423097 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 22:28:15.423097 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 22:28:15.423097 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 22:28:15.423097 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:28:15.423097 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 600/1195] 10.222.0.81 () {56 vars in 1096 bytes} [Wed Jul 20 22:28:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:28:15.428129 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1533711-7f60-4a12-8ff5-b63c8aea2080 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:28:15.522386 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1533711-7f60-4a12-8ff5-b63c8aea2080 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:28:15.555102 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3371a86-1fb9-414f-ab07-86deb13896e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:28:15.555806 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3371a86-1fb9-414f-ab07-86deb13896e6 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:28:15.556137 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3371a86-1fb9-414f-ab07-86deb13896e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:28:15.556416 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3371a86-1fb9-414f-ab07-86deb13896e6 None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:28:15.557150 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3371a86-1fb9-414f-ab07-86deb13896e6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:28:15.557456 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3371a86-1fb9-414f-ab07-86deb13896e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:28:15.558805 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3371a86-1fb9-414f-ab07-86deb13896e6 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:28:15.566554 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 596/1196] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Jul 20 22:28:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:28:15.572873 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0317f08e-0301-405e-9e0b-7e78388e1f3f None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:28:15.635614 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:28:15.673905 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0317f08e-0301-405e-9e0b-7e78388e1f3f None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:28:15.706978 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f668f8ad-26f9-4bc9-a40e-fb18479de72f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:28:15.707875 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f668f8ad-26f9-4bc9-a40e-fb18479de72f None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:28:15.708158 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f668f8ad-26f9-4bc9-a40e-fb18479de72f None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:28:15.708435 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f668f8ad-26f9-4bc9-a40e-fb18479de72f None admin] PATH_INFO: `/v3/users/fdad1f3206ea49d8bcabd5a8403b2cd6` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:28:15.709596 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f668f8ad-26f9-4bc9-a40e-fb18479de72f None admin] RBAC: Authorizing `identity:get_user(user_id=fdad1f3206ea49d8bcabd5a8403b2cd6)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:28:15.711043 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f668f8ad-26f9-4bc9-a40e-fb18479de72f None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:28:15.713425 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 601/1197] 10.222.0.81 () {56 vars in 1177 bytes} [Wed Jul 20 22:28:15 2022] GET /identity/v3/users/fdad1f3206ea49d8bcabd5a8403b2cd6 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:28:15.719197 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3371a86-1fb9-414f-ab07-86deb13896e6 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:28:15.722877 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:28:15.815962 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3371a86-1fb9-414f-ab07-86deb13896e6 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:28:15.847471 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575b073b-0fc4-4f37-83c9-02982788032e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:28:15.848114 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575b073b-0fc4-4f37-83c9-02982788032e None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:28:15.848399 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575b073b-0fc4-4f37-83c9-02982788032e None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:28:15.848676 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575b073b-0fc4-4f37-83c9-02982788032e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:28:15.913026 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575b073b-0fc4-4f37-83c9-02982788032e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:28:15.914450 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575b073b-0fc4-4f37-83c9-02982788032e None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:28:15.917602 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:28:16.009983 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 597/1198] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Jul 20 22:28:15 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 22:28:16.015912 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f668f8ad-26f9-4bc9-a40e-fb18479de72f None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:28:16.113741 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f668f8ad-26f9-4bc9-a40e-fb18479de72f None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:28:16.142597 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:28:16.147362 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759893f5-233d-487b-b5ad-a74f726ffeb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:28:16.148240 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759893f5-233d-487b-b5ad-a74f726ffeb5 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:28:16.148521 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759893f5-233d-487b-b5ad-a74f726ffeb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:28:16.148795 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759893f5-233d-487b-b5ad-a74f726ffeb5 None admin] PATH_INFO: `/v3/projects/35a4b0462b7f471eba8a159a67bc9d7e` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:28:16.150053 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759893f5-233d-487b-b5ad-a74f726ffeb5 None admin] RBAC: Authorizing `identity:get_project(project_id=35a4b0462b7f471eba8a159a67bc9d7e)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:28:16.151546 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759893f5-233d-487b-b5ad-a74f726ffeb5 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:28:16.158903 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 602/1199] 10.222.0.81 () {56 vars in 1186 bytes} [Wed Jul 20 22:28:16 2022] GET /identity/v3/projects/35a4b0462b7f471eba8a159a67bc9d7e => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:28:16.159736 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575b073b-0fc4-4f37-83c9-02982788032e None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:28:16.255221 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575b073b-0fc4-4f37-83c9-02982788032e None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:28:16.257452 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:28:16.287592 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2bc8f0-e672-406c-bf54-a1857f42fda7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:28:16.288180 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2bc8f0-e672-406c-bf54-a1857f42fda7 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:28:16.288463 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2bc8f0-e672-406c-bf54-a1857f42fda7 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:28:16.288816 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2bc8f0-e672-406c-bf54-a1857f42fda7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:28:16.289626 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2bc8f0-e672-406c-bf54-a1857f42fda7 None admin] RBAC: Adding query filter params (role.id=d89b6c92d35a401a8517f1d47f6be843, scope.project.id=35a4b0462b7f471eba8a159a67bc9d7e, user.id=fdad1f3206ea49d8bcabd5a8403b2cd6) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:28:16.290472 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2bc8f0-e672-406c-bf54-a1857f42fda7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:28:16.291900 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2bc8f0-e672-406c-bf54-a1857f42fda7 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:28:16.298284 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 598/1200] 10.222.0.81 () {56 vars in 1374 bytes} [Wed Jul 20 22:28:16 2022] GET /identity/v3/role_assignments?user.id=fdad1f3206ea49d8bcabd5a8403b2cd6&scope.project.id=35a4b0462b7f471eba8a159a67bc9d7e&role.id=d89b6c92d35a401a8517f1d47f6be843 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:28:26.296193 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=16 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:29:39.884571 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6767f6c-947b-4229-893b-e8d55e2f2fcf None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:29:39.885173 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6767f6c-947b-4229-893b-e8d55e2f2fcf None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:29:39.885173 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6767f6c-947b-4229-893b-e8d55e2f2fcf None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:29:39.886514 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 603/1201] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:29:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:29:39.891846 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d71796-3a80-4c19-a52f-69a84e09dfba None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:29:39.892307 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d71796-3a80-4c19-a52f-69a84e09dfba None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:29:39.892404 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d71796-3a80-4c19-a52f-69a84e09dfba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:29:39.909543 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-66d71796-3a80-4c19-a52f-69a84e09dfba None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:29:39.910720 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-66d71796-3a80-4c19-a52f-69a84e09dfba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:29:39.976150 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 599/1202] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:29:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 22:29:40.001895 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6767f6c-947b-4229-893b-e8d55e2f2fcf None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:29:40.104824 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6767f6c-947b-4229-893b-e8d55e2f2fcf None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:29:40.138133 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56293b9a-3f77-4dcf-8471-fb84f2eb0195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:29:40.138874 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56293b9a-3f77-4dcf-8471-fb84f2eb0195 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:29:40.139203 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56293b9a-3f77-4dcf-8471-fb84f2eb0195 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:29:40.139484 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56293b9a-3f77-4dcf-8471-fb84f2eb0195 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:29:40.149807 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56293b9a-3f77-4dcf-8471-fb84f2eb0195 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:29:40.156421 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56293b9a-3f77-4dcf-8471-fb84f2eb0195 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:29:40.169425 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-56293b9a-3f77-4dcf-8471-fb84f2eb0195 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:29:40.169425 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:29:40.169425 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:29:40.169425 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:29:40.169425 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:29:40.169425 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:29:40.169425 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:29:40.169425 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:29:40.169425 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:29:40.169425 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:29:40.169425 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:29:40.169425 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:29:40.169425 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 22:29:40.169425 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 22:29:40.169425 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 22:29:40.170551 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 22:29:40.170551 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:29:40.170551 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:29:40.170551 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:29:40.170551 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:29:40.170551 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:29:40.170551 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:29:40.170551 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:29:40.170551 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:29:40.170551 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:29:40.170551 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:29:40.170551 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:29:40.170551 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:29:40.170551 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:29:40.170551 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:29:40.170551 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:29:40.170551 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:29:40.171757 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 22:29:40.171757 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 22:29:40.171757 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:29:40.171757 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:29:40.171757 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 604/1203] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Jul 20 22:29:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:29:40.176938 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d71796-3a80-4c19-a52f-69a84e09dfba None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:29:40.274452 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d71796-3a80-4c19-a52f-69a84e09dfba None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:29:40.301686 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:29:40.307800 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef59a91c-ac62-4c8b-aca5-7f60bb677989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:29:40.308465 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef59a91c-ac62-4c8b-aca5-7f60bb677989 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:29:40.308754 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef59a91c-ac62-4c8b-aca5-7f60bb677989 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:29:40.309025 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef59a91c-ac62-4c8b-aca5-7f60bb677989 None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:29:40.309736 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef59a91c-ac62-4c8b-aca5-7f60bb677989 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:29:40.310041 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef59a91c-ac62-4c8b-aca5-7f60bb677989 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:29:40.311444 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef59a91c-ac62-4c8b-aca5-7f60bb677989 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:29:40.322994 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 600/1204] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Jul 20 22:29:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:29:40.329048 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56293b9a-3f77-4dcf-8471-fb84f2eb0195 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:29:40.381450 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:29:40.428963 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56293b9a-3f77-4dcf-8471-fb84f2eb0195 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:29:40.462388 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05d204a-d16a-4a7f-8d8e-02a76a749f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:29:40.463089 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05d204a-d16a-4a7f-8d8e-02a76a749f10 None admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:29:40.463478 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05d204a-d16a-4a7f-8d8e-02a76a749f10 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:29:40.463713 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05d204a-d16a-4a7f-8d8e-02a76a749f10 None admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:29:40.464434 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05d204a-d16a-4a7f-8d8e-02a76a749f10 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:29:40.465877 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05d204a-d16a-4a7f-8d8e-02a76a749f10 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:29:40.556583 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 605/1205] 10.222.0.81 () {60 vars in 1132 bytes} [Wed Jul 20 22:29:40 2022] POST /identity/v3/users => generated 251 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:29:42.625754 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b756d7c5-e106-47a4-9f97-bd3da5b9327e None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:29:42.626310 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b756d7c5-e106-47a4-9f97-bd3da5b9327e None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:29:42.626310 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b756d7c5-e106-47a4-9f97-bd3da5b9327e None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:29:42.627872 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 601/1206] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:29:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:29:42.633103 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5981d9de-ed42-4d22-a850-8a86c1eca11e None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:29:42.633499 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5981d9de-ed42-4d22-a850-8a86c1eca11e None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:29:42.633666 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5981d9de-ed42-4d22-a850-8a86c1eca11e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:29:42.651921 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-5981d9de-ed42-4d22-a850-8a86c1eca11e None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:29:42.653091 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-5981d9de-ed42-4d22-a850-8a86c1eca11e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:29:42.720039 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 606/1207] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:29:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 22:29:42.743294 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b756d7c5-e106-47a4-9f97-bd3da5b9327e None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:29:42.845735 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b756d7c5-e106-47a4-9f97-bd3da5b9327e None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:29:42.882810 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd971246-8bf4-4813-8778-f6befd323812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:29:42.882810 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd971246-8bf4-4813-8778-f6befd323812 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:29:42.882810 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd971246-8bf4-4813-8778-f6befd323812 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:29:42.882810 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd971246-8bf4-4813-8778-f6befd323812 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:29:42.891802 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd971246-8bf4-4813-8778-f6befd323812 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 22:29:42.900845 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-fd971246-8bf4-4813-8778-f6befd323812 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 22:29:42.900845 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:29:42.900845 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:29:42.900845 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:29:42.900845 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:29:42.900845 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:29:42.900845 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:29:42.900845 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:29:42.900845 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:29:42.900845 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:29:42.900845 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:29:42.900845 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:29:42.900845 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 22:29:42.900845 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 22:29:42.900845 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 22:29:42.900845 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 22:29:42.901805 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 22:29:42.901805 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 22:29:42.901805 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 22:29:42.901805 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 22:29:42.901805 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:29:42.901805 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:29:42.901805 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:29:42.901805 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:29:42.901805 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:29:42.901805 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:29:42.901805 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:29:42.901805 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:29:42.901805 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:29:42.901805 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:29:42.901805 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:29:42.901805 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:29:42.902699 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:29:42.902699 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:29:42.902699 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:29:42.902699 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:29:42.902699 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 22:29:42.902699 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 22:29:42.902699 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 22:29:42.902699 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 22:29:42.902699 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 22:29:42.902699 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 22:29:42.902699 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 22:29:42.902699 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:29:42.902699 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 602/1208] 10.222.0.81 () {56 vars in 1102 bytes} [Wed Jul 20 22:29:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:29:42.908182 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5981d9de-ed42-4d22-a850-8a86c1eca11e None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:29:42.947375 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:29:43.009993 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5981d9de-ed42-4d22-a850-8a86c1eca11e None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:29:43.044537 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8218a66-eb03-4498-a11d-4140911b7eb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:29:43.045206 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8218a66-eb03-4498-a11d-4140911b7eb2 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:29:43.045537 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8218a66-eb03-4498-a11d-4140911b7eb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:29:43.045821 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8218a66-eb03-4498-a11d-4140911b7eb2 None admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:29:43.046644 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8218a66-eb03-4498-a11d-4140911b7eb2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:29:43.046959 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8218a66-eb03-4498-a11d-4140911b7eb2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:29:43.048375 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8218a66-eb03-4498-a11d-4140911b7eb2 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:29:43.056077 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 607/1209] 10.222.0.81 () {56 vars in 1103 bytes} [Wed Jul 20 22:29:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:29:43.061930 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd971246-8bf4-4813-8778-f6befd323812 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:29:43.111834 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:29:43.160945 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd971246-8bf4-4813-8778-f6befd323812 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:29:43.197852 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5e17fa-550a-45ff-ba1f-d96157d9ddd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:29:43.199906 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5e17fa-550a-45ff-ba1f-d96157d9ddd6 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:29:43.199906 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5e17fa-550a-45ff-ba1f-d96157d9ddd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:29:43.199906 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5e17fa-550a-45ff-ba1f-d96157d9ddd6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:29:43.210266 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5e17fa-550a-45ff-ba1f-d96157d9ddd6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:29:43.211911 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5e17fa-550a-45ff-ba1f-d96157d9ddd6 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:29:43.229895 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-0f5e17fa-550a-45ff-ba1f-d96157d9ddd6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:29:43.229895 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:29:43.229895 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:29:43.229895 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:29:43.229895 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:29:43.229895 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:29:43.229895 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:29:43.229895 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:29:43.229895 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:29:43.229895 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:29:43.229895 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:29:43.229895 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:29:43.229895 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 22:29:43.229895 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 22:29:43.229895 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 22:29:43.230929 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 22:29:43.230929 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:29:43.230929 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:29:43.230929 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:29:43.230929 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:29:43.230929 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:29:43.230929 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:29:43.230929 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:29:43.230929 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:29:43.230929 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:29:43.230929 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:29:43.230929 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:29:43.230929 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:29:43.230929 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:29:43.230929 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:29:43.230929 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:29:43.230929 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:29:43.232063 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 22:29:43.232063 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 22:29:43.232063 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:29:43.232063 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:29:43.232063 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 603/1210] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Jul 20 22:29:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:29:43.237091 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8218a66-eb03-4498-a11d-4140911b7eb2 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:29:43.332624 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:29:43.335509 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8218a66-eb03-4498-a11d-4140911b7eb2 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:29:43.368168 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c192ce42-0034-40bc-8b7e-0f669da2d826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:29:43.368827 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c192ce42-0034-40bc-8b7e-0f669da2d826 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:29:43.369686 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c192ce42-0034-40bc-8b7e-0f669da2d826 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:29:43.369999 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c192ce42-0034-40bc-8b7e-0f669da2d826 None admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:29:43.370921 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c192ce42-0034-40bc-8b7e-0f669da2d826 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:29:43.373275 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c192ce42-0034-40bc-8b7e-0f669da2d826 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:29:43.373275 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c192ce42-0034-40bc-8b7e-0f669da2d826 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:29:43.384776 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 608/1211] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Jul 20 22:29:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:29:43.390612 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5e17fa-550a-45ff-ba1f-d96157d9ddd6 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:29:43.488761 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5e17fa-550a-45ff-ba1f-d96157d9ddd6 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:29:43.490191 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:29:43.521788 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8efb0c-68f6-4409-837a-12968df34c81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:29:43.522600 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8efb0c-68f6-4409-837a-12968df34c81 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:29:43.522885 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8efb0c-68f6-4409-837a-12968df34c81 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:29:43.523199 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8efb0c-68f6-4409-837a-12968df34c81 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:29:43.530688 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8efb0c-68f6-4409-837a-12968df34c81 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:29:43.533835 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8efb0c-68f6-4409-837a-12968df34c81 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:29:43.547628 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-ad8efb0c-68f6-4409-837a-12968df34c81 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 20 22:29:43.547628 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:29:43.547628 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:29:43.547628 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:29:43.547628 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:29:43.547628 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:29:43.547628 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:29:43.547628 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:29:43.547628 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:29:43.547628 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:29:43.547628 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:29:43.547628 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:29:43.547628 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 22:29:43.547628 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 22:29:43.547628 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 22:29:43.547628 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 22:29:43.548781 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:29:43.548781 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:29:43.548781 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 22:29:43.548781 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 22:29:43.548781 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 22:29:43.548781 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 22:29:43.548781 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:29:43.548781 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:29:43.548781 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:29:43.548781 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:29:43.548781 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:29:43.548781 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:29:43.548781 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:29:43.548781 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:29:43.548781 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:29:43.548781 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:29:43.548781 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:29:43.548781 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:29:43.549959 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:29:43.549959 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:29:43.549959 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 22:29:43.549959 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 22:29:43.549959 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 22:29:43.549959 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 22:29:43.549959 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 22:29:43.549959 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 22:29:43.549959 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 20 22:29:43.549959 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:29:43.549959 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 604/1212] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Jul 20 22:29:43 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:29:43.554671 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c192ce42-0034-40bc-8b7e-0f669da2d826 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:29:43.652928 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c192ce42-0034-40bc-8b7e-0f669da2d826 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:29:43.689642 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2228ad37-1b7e-48ec-a562-358c22b6d21f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:29:43.690332 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2228ad37-1b7e-48ec-a562-358c22b6d21f None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:29:43.690615 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2228ad37-1b7e-48ec-a562-358c22b6d21f None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:29:43.690890 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2228ad37-1b7e-48ec-a562-358c22b6d21f None admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:29:43.691752 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2228ad37-1b7e-48ec-a562-358c22b6d21f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:29:43.692060 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2228ad37-1b7e-48ec-a562-358c22b6d21f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:29:43.693517 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2228ad37-1b7e-48ec-a562-358c22b6d21f None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:29:43.729981 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 609/1213] 10.222.0.81 () {56 vars in 1143 bytes} [Wed Jul 20 22:29:43 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:29:43.736680 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8efb0c-68f6-4409-837a-12968df34c81 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:29:43.838542 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8efb0c-68f6-4409-837a-12968df34c81 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:29:43.872040 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab8c626-7b0c-4906-8f9f-77c9ba717fc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:29:43.872686 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab8c626-7b0c-4906-8f9f-77c9ba717fc0 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:29:43.872970 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab8c626-7b0c-4906-8f9f-77c9ba717fc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:29:43.873246 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab8c626-7b0c-4906-8f9f-77c9ba717fc0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:29:43.903836 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:29:43.939292 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab8c626-7b0c-4906-8f9f-77c9ba717fc0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:29:43.940794 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab8c626-7b0c-4906-8f9f-77c9ba717fc0 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:29:44.035878 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 605/1214] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Jul 20 22:29:43 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 22:29:44.041786 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2228ad37-1b7e-48ec-a562-358c22b6d21f None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:29:44.060614 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:29:44.141814 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2228ad37-1b7e-48ec-a562-358c22b6d21f None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:29:44.175368 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f25b93d-26a9-4473-8536-bb2c0df59b18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:29:44.176158 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f25b93d-26a9-4473-8536-bb2c0df59b18 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:29:44.176437 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f25b93d-26a9-4473-8536-bb2c0df59b18 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:29:44.176709 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f25b93d-26a9-4473-8536-bb2c0df59b18 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:29:44.187822 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f25b93d-26a9-4473-8536-bb2c0df59b18 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:29:44.189261 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f25b93d-26a9-4473-8536-bb2c0df59b18 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:29:44.209286 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-6f25b93d-26a9-4473-8536-bb2c0df59b18 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:29:44.209286 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:29:44.209286 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:29:44.209286 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:29:44.209286 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:29:44.209286 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:29:44.209286 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:29:44.209286 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:29:44.209286 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:29:44.209286 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:29:44.209286 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:29:44.209286 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:29:44.209286 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 22:29:44.209286 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 22:29:44.209286 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 22:29:44.210357 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 22:29:44.210357 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:29:44.210357 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:29:44.210357 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:29:44.210357 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:29:44.210357 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:29:44.210357 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:29:44.210357 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:29:44.210357 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:29:44.210357 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:29:44.210357 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:29:44.210357 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:29:44.210357 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:29:44.210357 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:29:44.210357 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:29:44.210357 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:29:44.210357 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:29:44.211493 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 22:29:44.211493 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 22:29:44.211493 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:29:44.211493 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:29:44.211493 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 610/1215] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Jul 20 22:29:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:29:44.217000 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab8c626-7b0c-4906-8f9f-77c9ba717fc0 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:29:44.276412 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:29:44.313910 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab8c626-7b0c-4906-8f9f-77c9ba717fc0 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:29:44.347635 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6431a4-ae88-48b6-8bb8-5b73c36570eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:29:44.348306 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6431a4-ae88-48b6-8bb8-5b73c36570eb None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:29:44.348595 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6431a4-ae88-48b6-8bb8-5b73c36570eb None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:29:44.348875 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6431a4-ae88-48b6-8bb8-5b73c36570eb None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:29:44.349707 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6431a4-ae88-48b6-8bb8-5b73c36570eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:29:44.350020 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6431a4-ae88-48b6-8bb8-5b73c36570eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:29:44.351437 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6431a4-ae88-48b6-8bb8-5b73c36570eb None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:29:44.363474 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 606/1216] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Jul 20 22:29:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:29:44.369154 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f25b93d-26a9-4473-8536-bb2c0df59b18 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:29:44.439568 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:29:44.469813 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f25b93d-26a9-4473-8536-bb2c0df59b18 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:29:44.503292 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f73293-9cdd-4f49-9977-798446615003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:29:44.504185 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f73293-9cdd-4f49-9977-798446615003 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:29:44.504473 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f73293-9cdd-4f49-9977-798446615003 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:29:44.504753 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f73293-9cdd-4f49-9977-798446615003 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:29:44.512115 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f73293-9cdd-4f49-9977-798446615003 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:29:44.513548 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f73293-9cdd-4f49-9977-798446615003 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:29:44.529629 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-12f73293-9cdd-4f49-9977-798446615003 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 22:29:44.529629 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:29:44.529629 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:29:44.529629 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:29:44.529629 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:29:44.529629 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:29:44.529629 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:29:44.529629 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:29:44.529629 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:29:44.529629 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:29:44.529629 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:29:44.529629 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:29:44.529629 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 22:29:44.529629 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 22:29:44.529629 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 22:29:44.530683 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 22:29:44.530683 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:29:44.530683 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:29:44.530683 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:29:44.530683 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:29:44.530683 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:29:44.530683 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:29:44.530683 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:29:44.530683 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:29:44.530683 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:29:44.530683 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:29:44.530683 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:29:44.530683 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:29:44.530683 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:29:44.530683 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:29:44.530683 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:29:44.530683 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:29:44.531891 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 22:29:44.531891 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 22:29:44.531891 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 22:29:44.531891 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 22:29:44.531891 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 22:29:44.531891 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 22:29:44.531891 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 22:29:44.531891 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:29:44.531891 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 611/1217] 10.222.0.81 () {56 vars in 1111 bytes} [Wed Jul 20 22:29:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:29:44.536883 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6431a4-ae88-48b6-8bb8-5b73c36570eb None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:29:44.635121 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6431a4-ae88-48b6-8bb8-5b73c36570eb None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:29:44.657122 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:29:44.668673 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c060faa-792a-424d-b78d-40e77031cce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:29:44.669308 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c060faa-792a-424d-b78d-40e77031cce4 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:29:44.669589 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c060faa-792a-424d-b78d-40e77031cce4 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:29:44.669952 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c060faa-792a-424d-b78d-40e77031cce4 None admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:29:44.670759 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c060faa-792a-424d-b78d-40e77031cce4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:29:44.671125 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c060faa-792a-424d-b78d-40e77031cce4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:29:44.672556 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c060faa-792a-424d-b78d-40e77031cce4 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:29:44.684740 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 607/1218] 10.222.0.81 () {56 vars in 1148 bytes} [Wed Jul 20 22:29:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:29:44.690988 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f73293-9cdd-4f49-9977-798446615003 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:29:44.805937 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f73293-9cdd-4f49-9977-798446615003 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:29:44.839827 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:29:44.845311 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cbe473-5d25-4205-a887-d3f40e17721e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:29:44.845965 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cbe473-5d25-4205-a887-d3f40e17721e None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:29:44.846253 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cbe473-5d25-4205-a887-d3f40e17721e None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:29:44.846534 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cbe473-5d25-4205-a887-d3f40e17721e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:29:44.847383 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cbe473-5d25-4205-a887-d3f40e17721e None admin] RBAC: Adding query filter params (role.id=9b4a83e2eca44a8da9f356bc40f8788a, scope.project.id=385edf2e59e546d090ace36600e29032, user.id=d5500cebdd9d4f8e9aecb45d6c955878) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:29:44.847700 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cbe473-5d25-4205-a887-d3f40e17721e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:29:44.849125 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cbe473-5d25-4205-a887-d3f40e17721e None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:29:44.854412 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 612/1219] 10.222.0.81 () {56 vars in 1374 bytes} [Wed Jul 20 22:29:44 2022] GET /identity/v3/role_assignments?user.id=d5500cebdd9d4f8e9aecb45d6c955878&scope.project.id=385edf2e59e546d090ace36600e29032&role.id=9b4a83e2eca44a8da9f356bc40f8788a => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:29:46.892939 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817573b1-2ee7-4779-abfa-ace683faa50f None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:29:46.893471 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817573b1-2ee7-4779-abfa-ace683faa50f None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:29:46.893579 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817573b1-2ee7-4779-abfa-ace683faa50f None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:29:46.894926 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 608/1220] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:29:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:29:46.900418 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac85bd9-2353-4f30-8e36-f9d4fc1f17e7 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:29:46.900821 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac85bd9-2353-4f30-8e36-f9d4fc1f17e7 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:29:46.900982 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac85bd9-2353-4f30-8e36-f9d4fc1f17e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:29:46.917996 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-bac85bd9-2353-4f30-8e36-f9d4fc1f17e7 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:29:46.919288 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-bac85bd9-2353-4f30-8e36-f9d4fc1f17e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:29:46.984920 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 613/1221] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:29:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 22:29:47.007365 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817573b1-2ee7-4779-abfa-ace683faa50f None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:29:47.111607 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817573b1-2ee7-4779-abfa-ace683faa50f None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:29:47.145647 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a5fffc-78f0-47d1-a5c7-5a179edf9444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:29:47.146226 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a5fffc-78f0-47d1-a5c7-5a179edf9444 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:29:47.146514 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a5fffc-78f0-47d1-a5c7-5a179edf9444 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:29:47.146791 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a5fffc-78f0-47d1-a5c7-5a179edf9444 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:29:47.157535 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a5fffc-78f0-47d1-a5c7-5a179edf9444 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 22:29:47.165748 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-44a5fffc-78f0-47d1-a5c7-5a179edf9444 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 22:29:47.165748 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:29:47.165748 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:29:47.165748 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:29:47.165748 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:29:47.165748 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:29:47.165748 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:29:47.165748 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:29:47.165748 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:29:47.165748 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:29:47.165748 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:29:47.165748 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:29:47.165748 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 22:29:47.165748 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 22:29:47.165748 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 22:29:47.165748 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 22:29:47.166826 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 22:29:47.166826 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 22:29:47.166826 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 22:29:47.166826 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 22:29:47.166826 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:29:47.166826 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:29:47.166826 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:29:47.166826 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:29:47.166826 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:29:47.166826 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:29:47.166826 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:29:47.166826 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:29:47.166826 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:29:47.166826 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:29:47.166826 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:29:47.166826 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:29:47.168500 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:29:47.168500 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:29:47.168500 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:29:47.168500 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:29:47.168500 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 22:29:47.168500 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 22:29:47.168500 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 22:29:47.168500 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 22:29:47.168500 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 22:29:47.168500 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 22:29:47.168500 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 22:29:47.168500 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:29:47.168500 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 609/1222] 10.222.0.81 () {56 vars in 1102 bytes} [Wed Jul 20 22:29:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:29:47.173955 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac85bd9-2353-4f30-8e36-f9d4fc1f17e7 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:29:47.271560 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac85bd9-2353-4f30-8e36-f9d4fc1f17e7 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:29:47.296040 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:29:47.306516 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c0d499-20f7-4590-8f58-b85e63997f55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:29:47.307957 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c0d499-20f7-4590-8f58-b85e63997f55 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:29:47.307957 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c0d499-20f7-4590-8f58-b85e63997f55 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:29:47.307957 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c0d499-20f7-4590-8f58-b85e63997f55 None admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:29:47.308625 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c0d499-20f7-4590-8f58-b85e63997f55 None admin] RBAC: Adding query filter params (name=service) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:29:47.308933 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c0d499-20f7-4590-8f58-b85e63997f55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:29:47.310297 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c0d499-20f7-4590-8f58-b85e63997f55 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:29:47.318109 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 614/1223] 10.222.0.81 () {56 vars in 1103 bytes} [Wed Jul 20 22:29:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:29:47.324225 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a5fffc-78f0-47d1-a5c7-5a179edf9444 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:29:47.367375 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:29:47.423785 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a5fffc-78f0-47d1-a5c7-5a179edf9444 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:29:47.471152 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37571008-9a8d-4ab5-b83f-48126903ab52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:29:47.471152 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37571008-9a8d-4ab5-b83f-48126903ab52 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:29:47.471152 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37571008-9a8d-4ab5-b83f-48126903ab52 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:29:47.471152 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37571008-9a8d-4ab5-b83f-48126903ab52 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:29:47.471785 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37571008-9a8d-4ab5-b83f-48126903ab52 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:29:47.471785 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37571008-9a8d-4ab5-b83f-48126903ab52 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:29:47.487407 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-37571008-9a8d-4ab5-b83f-48126903ab52 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:29:47.487407 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:29:47.487407 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:29:47.487407 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:29:47.487407 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:29:47.487407 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:29:47.487407 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:29:47.487407 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:29:47.487407 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:29:47.487407 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:29:47.487407 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:29:47.487407 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:29:47.487407 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 22:29:47.487407 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 22:29:47.487407 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 22:29:47.488648 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 22:29:47.488648 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:29:47.488648 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:29:47.488648 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:29:47.488648 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:29:47.488648 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:29:47.488648 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:29:47.488648 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:29:47.488648 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:29:47.488648 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:29:47.488648 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:29:47.488648 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:29:47.488648 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:29:47.488648 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:29:47.488648 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:29:47.488648 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:29:47.488648 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:29:47.489882 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 22:29:47.489882 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 22:29:47.489882 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:29:47.489882 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:29:47.489882 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 610/1224] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Jul 20 22:29:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:29:47.494683 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c0d499-20f7-4590-8f58-b85e63997f55 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:29:47.594600 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c0d499-20f7-4590-8f58-b85e63997f55 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:29:47.628150 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29ebaef-f9ad-492e-8706-93337ca7a736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:29:47.628803 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29ebaef-f9ad-492e-8706-93337ca7a736 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:29:47.629085 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29ebaef-f9ad-492e-8706-93337ca7a736 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:29:47.629363 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29ebaef-f9ad-492e-8706-93337ca7a736 None admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:29:47.630069 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29ebaef-f9ad-492e-8706-93337ca7a736 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:29:47.630373 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29ebaef-f9ad-492e-8706-93337ca7a736 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:29:47.631843 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29ebaef-f9ad-492e-8706-93337ca7a736 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:29:47.644443 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 615/1225] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Jul 20 22:29:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:29:47.650447 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37571008-9a8d-4ab5-b83f-48126903ab52 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:29:47.741979 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:29:47.749018 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37571008-9a8d-4ab5-b83f-48126903ab52 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:29:47.780953 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cc916e-0df4-44cb-ac88-bfc0c5dcbfc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:29:47.781772 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cc916e-0df4-44cb-ac88-bfc0c5dcbfc9 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:29:47.782060 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cc916e-0df4-44cb-ac88-bfc0c5dcbfc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:29:47.782339 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cc916e-0df4-44cb-ac88-bfc0c5dcbfc9 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:29:47.789715 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cc916e-0df4-44cb-ac88-bfc0c5dcbfc9 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:29:47.791189 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cc916e-0df4-44cb-ac88-bfc0c5dcbfc9 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:29:47.806233 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-73cc916e-0df4-44cb-ac88-bfc0c5dcbfc9 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 20 22:29:47.806233 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:29:47.806233 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:29:47.806233 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:29:47.806233 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:29:47.806233 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:29:47.806233 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:29:47.806233 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:29:47.806233 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:29:47.806233 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:29:47.806233 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:29:47.806233 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:29:47.806233 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 22:29:47.806233 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 22:29:47.806233 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 22:29:47.806233 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 22:29:47.807355 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:29:47.807355 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:29:47.807355 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 22:29:47.807355 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 22:29:47.807355 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 22:29:47.807355 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 22:29:47.807355 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:29:47.807355 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:29:47.807355 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:29:47.807355 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:29:47.807355 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:29:47.807355 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:29:47.807355 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:29:47.807355 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:29:47.807355 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:29:47.807355 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:29:47.807355 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:29:47.807355 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:29:47.808903 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:29:47.808903 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:29:47.808903 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 22:29:47.808903 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 22:29:47.808903 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 22:29:47.808903 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 22:29:47.808903 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 22:29:47.808903 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 22:29:47.808903 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 20 22:29:47.808903 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:29:47.808903 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 611/1226] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Jul 20 22:29:47 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:29:47.814129 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29ebaef-f9ad-492e-8706-93337ca7a736 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:29:47.860855 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:29:47.912873 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29ebaef-f9ad-492e-8706-93337ca7a736 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:29:47.945392 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bede336e-a9c5-4693-a411-b17381a81fc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:29:47.946054 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bede336e-a9c5-4693-a411-b17381a81fc7 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:29:47.946338 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bede336e-a9c5-4693-a411-b17381a81fc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:29:47.946615 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bede336e-a9c5-4693-a411-b17381a81fc7 None admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:29:47.947475 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bede336e-a9c5-4693-a411-b17381a81fc7 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:29:47.947814 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bede336e-a9c5-4693-a411-b17381a81fc7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:29:47.949299 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bede336e-a9c5-4693-a411-b17381a81fc7 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:29:47.966363 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 616/1227] 10.222.0.81 () {56 vars in 1143 bytes} [Wed Jul 20 22:29:47 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:29:47.972143 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cc916e-0df4-44cb-ac88-bfc0c5dcbfc9 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:29:48.070962 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cc916e-0df4-44cb-ac88-bfc0c5dcbfc9 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:29:48.105541 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931c2fd4-c00b-42be-8dc1-7870ad75f90a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:29:48.106382 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931c2fd4-c00b-42be-8dc1-7870ad75f90a None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:29:48.106665 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931c2fd4-c00b-42be-8dc1-7870ad75f90a None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:29:48.106939 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931c2fd4-c00b-42be-8dc1-7870ad75f90a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:29:48.117450 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931c2fd4-c00b-42be-8dc1-7870ad75f90a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:29:48.118872 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931c2fd4-c00b-42be-8dc1-7870ad75f90a None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:29:48.136142 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-931c2fd4-c00b-42be-8dc1-7870ad75f90a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:29:48.136142 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:29:48.136142 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:29:48.136142 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:29:48.136142 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:29:48.136142 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:29:48.136142 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:29:48.136142 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:29:48.136142 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:29:48.136142 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:29:48.136142 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:29:48.136142 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:29:48.136142 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 22:29:48.136142 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 22:29:48.136142 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 22:29:48.137106 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 22:29:48.137106 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:29:48.137106 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:29:48.137106 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:29:48.137106 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:29:48.137106 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:29:48.137106 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:29:48.137106 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:29:48.137106 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:29:48.137106 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:29:48.137106 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:29:48.137106 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:29:48.137106 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:29:48.137106 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:29:48.137106 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:29:48.137106 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:29:48.137106 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:29:48.138299 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 22:29:48.138299 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 22:29:48.138299 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:29:48.138299 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:29:48.138299 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 612/1228] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Jul 20 22:29:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:29:48.143160 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bede336e-a9c5-4693-a411-b17381a81fc7 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:29:48.243772 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bede336e-a9c5-4693-a411-b17381a81fc7 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:29:48.247278 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:29:48.278562 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67427c0-d2f1-403c-8cfe-5db88f5b26a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:29:48.281705 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67427c0-d2f1-403c-8cfe-5db88f5b26a5 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:29:48.281705 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67427c0-d2f1-403c-8cfe-5db88f5b26a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:29:48.281705 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67427c0-d2f1-403c-8cfe-5db88f5b26a5 None admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:29:48.281705 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67427c0-d2f1-403c-8cfe-5db88f5b26a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:29:48.281705 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67427c0-d2f1-403c-8cfe-5db88f5b26a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:29:48.282173 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67427c0-d2f1-403c-8cfe-5db88f5b26a5 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:29:48.293170 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 617/1229] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Jul 20 22:29:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:29:48.298792 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931c2fd4-c00b-42be-8dc1-7870ad75f90a None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:29:48.308017 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:29:48.395241 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931c2fd4-c00b-42be-8dc1-7870ad75f90a None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:29:48.427320 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e5b100-cfc9-4657-938b-f5423da8d304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:29:48.428121 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e5b100-cfc9-4657-938b-f5423da8d304 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:29:48.428407 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e5b100-cfc9-4657-938b-f5423da8d304 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:29:48.428681 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e5b100-cfc9-4657-938b-f5423da8d304 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:29:48.435027 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e5b100-cfc9-4657-938b-f5423da8d304 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:29:48.436462 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e5b100-cfc9-4657-938b-f5423da8d304 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:29:48.452128 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-16e5b100-cfc9-4657-938b-f5423da8d304 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 22:29:48.452128 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:29:48.452128 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:29:48.452128 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:29:48.452128 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:29:48.452128 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:29:48.452128 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:29:48.452128 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:29:48.452128 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:29:48.452128 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:29:48.452128 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:29:48.452128 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:29:48.452128 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 22:29:48.452128 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 22:29:48.452128 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 22:29:48.453194 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 22:29:48.453194 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:29:48.453194 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:29:48.453194 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:29:48.453194 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:29:48.453194 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:29:48.453194 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:29:48.453194 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:29:48.453194 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:29:48.453194 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:29:48.453194 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:29:48.453194 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:29:48.453194 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:29:48.453194 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:29:48.453194 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:29:48.453194 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:29:48.453194 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:29:48.454250 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 22:29:48.454250 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 22:29:48.454250 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 22:29:48.454250 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 22:29:48.454250 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 22:29:48.454250 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 22:29:48.454250 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 22:29:48.454250 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:29:48.454250 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 613/1230] 10.222.0.81 () {56 vars in 1111 bytes} [Wed Jul 20 22:29:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:29:48.459235 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67427c0-d2f1-403c-8cfe-5db88f5b26a5 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:29:48.562130 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67427c0-d2f1-403c-8cfe-5db88f5b26a5 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:29:48.594872 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b126199-8239-4f03-ac33-d337de974af8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:29:48.595560 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b126199-8239-4f03-ac33-d337de974af8 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:29:48.595850 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b126199-8239-4f03-ac33-d337de974af8 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:29:48.596129 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b126199-8239-4f03-ac33-d337de974af8 None admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:29:48.596953 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b126199-8239-4f03-ac33-d337de974af8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:29:48.597302 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b126199-8239-4f03-ac33-d337de974af8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:29:48.598784 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b126199-8239-4f03-ac33-d337de974af8 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:29:48.611935 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 618/1231] 10.222.0.81 () {56 vars in 1148 bytes} [Wed Jul 20 22:29:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:29:48.617990 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e5b100-cfc9-4657-938b-f5423da8d304 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:29:48.684175 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:29:48.717107 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e5b100-cfc9-4657-938b-f5423da8d304 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:29:48.749898 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81af8c6b-0de3-4d45-9cd7-e595f57cb38b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:29:48.750772 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81af8c6b-0de3-4d45-9cd7-e595f57cb38b None admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:29:48.751140 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81af8c6b-0de3-4d45-9cd7-e595f57cb38b None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:29:48.751418 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81af8c6b-0de3-4d45-9cd7-e595f57cb38b None admin] PATH_INFO: `/v3/projects/385edf2e59e546d090ace36600e29032/users/d5500cebdd9d4f8e9aecb45d6c955878/roles/9b4a83e2eca44a8da9f356bc40f8788a` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:29:48.765251 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81af8c6b-0de3-4d45-9cd7-e595f57cb38b None admin] RBAC: Authorizing `identity:create_grant(project_id=385edf2e59e546d090ace36600e29032, user_id=d5500cebdd9d4f8e9aecb45d6c955878, role_id=9b4a83e2eca44a8da9f356bc40f8788a)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:29:48.766716 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81af8c6b-0de3-4d45-9cd7-e595f57cb38b None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:29:48.895444 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 614/1232] 10.222.0.81 () {58 vars in 1439 bytes} [Wed Jul 20 22:29:48 2022] PUT /identity/v3/projects/385edf2e59e546d090ace36600e29032/users/d5500cebdd9d4f8e9aecb45d6c955878/roles/9b4a83e2eca44a8da9f356bc40f8788a => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:29:50.914134 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fd4107-c731-4672-b9a0-61f045ab3206 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:29:50.914706 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fd4107-c731-4672-b9a0-61f045ab3206 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:29:50.914706 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fd4107-c731-4672-b9a0-61f045ab3206 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:29:50.916241 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 619/1233] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:29:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:29:50.921771 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be383b4b-f653-4209-8bfd-a39a1e23abf5 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:29:50.922195 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be383b4b-f653-4209-8bfd-a39a1e23abf5 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:29:50.922381 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be383b4b-f653-4209-8bfd-a39a1e23abf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:29:50.939214 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-be383b4b-f653-4209-8bfd-a39a1e23abf5 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:29:50.940427 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-be383b4b-f653-4209-8bfd-a39a1e23abf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:29:51.004656 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 615/1234] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:29:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 22:29:51.028957 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fd4107-c731-4672-b9a0-61f045ab3206 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:29:51.055141 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:29:51.132459 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fd4107-c731-4672-b9a0-61f045ab3206 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:29:51.166854 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb276c3-472f-4c19-89c4-d184afcaae83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:29:51.167667 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb276c3-472f-4c19-89c4-d184afcaae83 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:29:51.167936 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb276c3-472f-4c19-89c4-d184afcaae83 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:29:51.168956 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb276c3-472f-4c19-89c4-d184afcaae83 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:29:51.178722 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb276c3-472f-4c19-89c4-d184afcaae83 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 22:29:51.190894 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-8fb276c3-472f-4c19-89c4-d184afcaae83 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 22:29:51.190894 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:29:51.190894 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:29:51.190894 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:29:51.190894 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:29:51.190894 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:29:51.190894 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:29:51.190894 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:29:51.190894 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:29:51.190894 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:29:51.190894 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:29:51.190894 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:29:51.190894 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 22:29:51.190894 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 22:29:51.190894 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 22:29:51.190894 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 22:29:51.191986 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 22:29:51.191986 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 22:29:51.191986 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 22:29:51.191986 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 22:29:51.191986 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:29:51.191986 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:29:51.191986 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:29:51.191986 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:29:51.191986 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:29:51.191986 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:29:51.191986 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:29:51.191986 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:29:51.191986 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:29:51.191986 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:29:51.191986 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:29:51.191986 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:29:51.192837 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:29:51.192837 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:29:51.192837 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:29:51.192837 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:29:51.192837 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 22:29:51.192837 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 22:29:51.192837 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 22:29:51.192837 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 22:29:51.192837 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 22:29:51.192837 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 22:29:51.192837 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 22:29:51.192837 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:29:51.192837 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 620/1235] 10.222.0.81 () {56 vars in 1102 bytes} [Wed Jul 20 22:29:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:29:51.198441 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be383b4b-f653-4209-8bfd-a39a1e23abf5 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:29:51.247191 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:29:51.299066 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be383b4b-f653-4209-8bfd-a39a1e23abf5 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:29:51.331282 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7196c4d-b389-49d6-9dc4-fe94d7c468ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:29:51.331980 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7196c4d-b389-49d6-9dc4-fe94d7c468ed None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:29:51.332266 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7196c4d-b389-49d6-9dc4-fe94d7c468ed None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:29:51.332547 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7196c4d-b389-49d6-9dc4-fe94d7c468ed None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:29:51.333287 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7196c4d-b389-49d6-9dc4-fe94d7c468ed None admin] RBAC: Adding query filter params (name=service) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:29:51.333596 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7196c4d-b389-49d6-9dc4-fe94d7c468ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:29:51.334983 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7196c4d-b389-49d6-9dc4-fe94d7c468ed None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:29:51.343810 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 616/1236] 10.222.0.81 () {56 vars in 1103 bytes} [Wed Jul 20 22:29:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:29:51.349728 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb276c3-472f-4c19-89c4-d184afcaae83 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:29:51.435408 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:29:51.451424 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb276c3-472f-4c19-89c4-d184afcaae83 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:29:51.485368 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7443c537-2876-4de6-a580-f27a74e125d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:29:51.486156 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7443c537-2876-4de6-a580-f27a74e125d3 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:29:51.486437 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7443c537-2876-4de6-a580-f27a74e125d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:29:51.486712 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7443c537-2876-4de6-a580-f27a74e125d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:29:51.497413 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7443c537-2876-4de6-a580-f27a74e125d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:29:51.498903 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7443c537-2876-4de6-a580-f27a74e125d3 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:29:51.525869 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-7443c537-2876-4de6-a580-f27a74e125d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:29:51.525869 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:29:51.525869 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:29:51.525869 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:29:51.525869 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:29:51.525869 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:29:51.525869 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:29:51.525869 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:29:51.525869 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:29:51.525869 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:29:51.525869 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:29:51.525869 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:29:51.525869 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 22:29:51.525869 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 22:29:51.525869 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 22:29:51.526930 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 22:29:51.526930 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:29:51.526930 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:29:51.526930 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:29:51.526930 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:29:51.526930 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:29:51.526930 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:29:51.526930 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:29:51.526930 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:29:51.526930 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:29:51.526930 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:29:51.526930 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:29:51.526930 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:29:51.526930 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:29:51.526930 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:29:51.526930 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:29:51.526930 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:29:51.527929 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 22:29:51.527929 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 22:29:51.527929 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:29:51.527929 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:29:51.527929 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 621/1237] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Jul 20 22:29:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:29:51.533225 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7196c4d-b389-49d6-9dc4-fe94d7c468ed None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:29:51.634978 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7196c4d-b389-49d6-9dc4-fe94d7c468ed None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:29:51.643927 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:29:51.669259 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d918d7d9-3660-4d61-8438-2284f9531393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:29:51.669927 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d918d7d9-3660-4d61-8438-2284f9531393 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:29:51.670212 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d918d7d9-3660-4d61-8438-2284f9531393 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:29:51.670489 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d918d7d9-3660-4d61-8438-2284f9531393 None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:29:51.671265 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d918d7d9-3660-4d61-8438-2284f9531393 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:29:51.671575 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d918d7d9-3660-4d61-8438-2284f9531393 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:29:51.672933 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d918d7d9-3660-4d61-8438-2284f9531393 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:29:51.683969 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 617/1238] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Jul 20 22:29:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:29:51.693596 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7443c537-2876-4de6-a580-f27a74e125d3 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:29:51.786835 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7443c537-2876-4de6-a580-f27a74e125d3 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:29:51.819977 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803ff1ac-b186-4f4c-ba92-c268772ceb4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:29:51.821362 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803ff1ac-b186-4f4c-ba92-c268772ceb4d None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:29:51.821362 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803ff1ac-b186-4f4c-ba92-c268772ceb4d None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:29:51.821598 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803ff1ac-b186-4f4c-ba92-c268772ceb4d None admin] PATH_INFO: `/v3/users/placement` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:29:51.824331 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:29:51.828844 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803ff1ac-b186-4f4c-ba92-c268772ceb4d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:29:51.830282 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803ff1ac-b186-4f4c-ba92-c268772ceb4d None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:29:51.845695 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-803ff1ac-b186-4f4c-ba92-c268772ceb4d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 20 22:29:51.845695 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:29:51.845695 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:29:51.845695 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:29:51.845695 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:29:51.845695 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:29:51.845695 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:29:51.845695 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:29:51.845695 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:29:51.845695 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:29:51.845695 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:29:51.845695 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:29:51.845695 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 22:29:51.845695 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 22:29:51.845695 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 22:29:51.845695 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 22:29:51.846821 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:29:51.846821 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:29:51.846821 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 22:29:51.846821 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 22:29:51.846821 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 22:29:51.846821 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 22:29:51.846821 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:29:51.846821 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:29:51.846821 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:29:51.846821 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:29:51.846821 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:29:51.846821 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:29:51.846821 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:29:51.846821 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:29:51.846821 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:29:51.846821 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:29:51.846821 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:29:51.846821 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:29:51.848084 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:29:51.848084 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:29:51.848084 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 22:29:51.848084 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 22:29:51.848084 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 22:29:51.848084 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 22:29:51.848084 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 22:29:51.848084 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 22:29:51.848084 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 20 22:29:51.848084 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:29:51.848084 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 622/1239] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Jul 20 22:29:51 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:29:51.852611 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d918d7d9-3660-4d61-8438-2284f9531393 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:29:51.949690 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d918d7d9-3660-4d61-8438-2284f9531393 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:29:51.981767 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db25e16-0098-440e-9a7e-3d8cc4d4026b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:29:51.982467 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db25e16-0098-440e-9a7e-3d8cc4d4026b None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:29:51.982752 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db25e16-0098-440e-9a7e-3d8cc4d4026b None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:29:51.983047 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db25e16-0098-440e-9a7e-3d8cc4d4026b None admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:29:51.983876 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db25e16-0098-440e-9a7e-3d8cc4d4026b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:29:51.984185 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db25e16-0098-440e-9a7e-3d8cc4d4026b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:29:51.985599 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db25e16-0098-440e-9a7e-3d8cc4d4026b None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:29:52.002717 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 618/1240] 10.222.0.81 () {56 vars in 1143 bytes} [Wed Jul 20 22:29:51 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:29:52.010428 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803ff1ac-b186-4f4c-ba92-c268772ceb4d None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:29:52.106625 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803ff1ac-b186-4f4c-ba92-c268772ceb4d None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:29:52.140117 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50de6ff-62b0-4f27-934e-ea25cabca968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:29:52.140754 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50de6ff-62b0-4f27-934e-ea25cabca968 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:29:52.141039 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50de6ff-62b0-4f27-934e-ea25cabca968 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:29:52.141318 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50de6ff-62b0-4f27-934e-ea25cabca968 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:29:52.201422 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:29:52.207874 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50de6ff-62b0-4f27-934e-ea25cabca968 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:29:52.209316 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50de6ff-62b0-4f27-934e-ea25cabca968 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:29:52.305749 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 623/1241] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Jul 20 22:29:52 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 22:29:52.311641 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db25e16-0098-440e-9a7e-3d8cc4d4026b None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:29:52.338823 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:29:52.410094 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db25e16-0098-440e-9a7e-3d8cc4d4026b None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:29:52.443894 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27af035b-a8f4-4377-ba51-346f3e7de78e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:29:52.444691 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27af035b-a8f4-4377-ba51-346f3e7de78e None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:29:52.444992 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27af035b-a8f4-4377-ba51-346f3e7de78e None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:29:52.445286 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27af035b-a8f4-4377-ba51-346f3e7de78e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:29:52.455741 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27af035b-a8f4-4377-ba51-346f3e7de78e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:29:52.457157 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27af035b-a8f4-4377-ba51-346f3e7de78e None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:29:52.475305 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-27af035b-a8f4-4377-ba51-346f3e7de78e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:29:52.475305 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:29:52.475305 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:29:52.475305 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:29:52.475305 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:29:52.475305 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:29:52.475305 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:29:52.475305 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:29:52.475305 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:29:52.475305 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:29:52.475305 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:29:52.475305 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:29:52.475305 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 22:29:52.475305 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 22:29:52.475305 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 22:29:52.476395 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 22:29:52.476395 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:29:52.476395 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:29:52.476395 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:29:52.476395 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:29:52.476395 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:29:52.476395 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:29:52.476395 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:29:52.476395 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:29:52.476395 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:29:52.476395 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:29:52.476395 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:29:52.476395 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:29:52.476395 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:29:52.476395 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:29:52.476395 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:29:52.476395 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:29:52.477444 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 22:29:52.477444 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 22:29:52.477444 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:29:52.477444 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:29:52.477444 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 619/1242] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Jul 20 22:29:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:29:52.482733 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50de6ff-62b0-4f27-934e-ea25cabca968 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:29:52.576642 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:29:52.582318 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50de6ff-62b0-4f27-934e-ea25cabca968 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:29:52.617569 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8409345c-efae-49bc-81f8-c2f58f85380c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:29:52.618219 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8409345c-efae-49bc-81f8-c2f58f85380c None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:29:52.618498 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8409345c-efae-49bc-81f8-c2f58f85380c None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:29:52.618771 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8409345c-efae-49bc-81f8-c2f58f85380c None admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:29:52.619511 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8409345c-efae-49bc-81f8-c2f58f85380c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:29:52.619852 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8409345c-efae-49bc-81f8-c2f58f85380c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:29:52.621295 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8409345c-efae-49bc-81f8-c2f58f85380c None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:29:52.633275 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 624/1243] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Jul 20 22:29:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:29:52.639585 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27af035b-a8f4-4377-ba51-346f3e7de78e None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:29:52.716174 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:29:52.737121 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27af035b-a8f4-4377-ba51-346f3e7de78e None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:29:52.770715 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b587b571-be3d-41fe-81bf-01ba6ab628a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:29:52.771545 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b587b571-be3d-41fe-81bf-01ba6ab628a9 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:29:52.771830 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b587b571-be3d-41fe-81bf-01ba6ab628a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:29:52.772109 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b587b571-be3d-41fe-81bf-01ba6ab628a9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:29:52.778926 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b587b571-be3d-41fe-81bf-01ba6ab628a9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:29:52.780383 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b587b571-be3d-41fe-81bf-01ba6ab628a9 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:29:52.794826 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-b587b571-be3d-41fe-81bf-01ba6ab628a9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 22:29:52.794826 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:29:52.794826 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:29:52.794826 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:29:52.794826 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:29:52.794826 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:29:52.794826 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:29:52.794826 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:29:52.794826 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:29:52.794826 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:29:52.794826 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:29:52.794826 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:29:52.794826 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 22:29:52.794826 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 22:29:52.794826 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 22:29:52.795912 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 22:29:52.795912 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:29:52.795912 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:29:52.795912 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:29:52.795912 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:29:52.795912 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:29:52.795912 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:29:52.795912 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:29:52.795912 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:29:52.795912 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:29:52.795912 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:29:52.795912 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:29:52.795912 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:29:52.795912 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:29:52.795912 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:29:52.795912 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:29:52.795912 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:29:52.797168 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 22:29:52.797168 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 22:29:52.797168 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 22:29:52.797168 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 22:29:52.797168 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 22:29:52.797168 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 22:29:52.797168 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 22:29:52.797168 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:29:52.797168 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 620/1244] 10.222.0.81 () {56 vars in 1111 bytes} [Wed Jul 20 22:29:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:29:52.801873 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8409345c-efae-49bc-81f8-c2f58f85380c None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:29:52.898675 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8409345c-efae-49bc-81f8-c2f58f85380c None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:29:52.931429 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f5a169-56e9-4f60-ba54-bf1bd5f57b88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:29:52.932080 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f5a169-56e9-4f60-ba54-bf1bd5f57b88 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:29:52.932363 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f5a169-56e9-4f60-ba54-bf1bd5f57b88 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:29:52.932640 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f5a169-56e9-4f60-ba54-bf1bd5f57b88 None admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:29:52.933412 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f5a169-56e9-4f60-ba54-bf1bd5f57b88 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:29:52.933723 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f5a169-56e9-4f60-ba54-bf1bd5f57b88 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:29:52.935763 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f5a169-56e9-4f60-ba54-bf1bd5f57b88 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:29:52.947397 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 625/1245] 10.222.0.81 () {56 vars in 1148 bytes} [Wed Jul 20 22:29:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:29:52.953641 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b587b571-be3d-41fe-81bf-01ba6ab628a9 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:29:53.051386 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b587b571-be3d-41fe-81bf-01ba6ab628a9 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:29:53.083921 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0173599-b2fd-486b-a2d5-47ab17bcc293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:29:53.084576 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0173599-b2fd-486b-a2d5-47ab17bcc293 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:29:53.084865 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0173599-b2fd-486b-a2d5-47ab17bcc293 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:29:53.085145 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0173599-b2fd-486b-a2d5-47ab17bcc293 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:29:53.085965 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0173599-b2fd-486b-a2d5-47ab17bcc293 None admin] RBAC: Adding query filter params (role.id=9b4a83e2eca44a8da9f356bc40f8788a, scope.project.id=385edf2e59e546d090ace36600e29032, user.id=d5500cebdd9d4f8e9aecb45d6c955878) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:29:53.086280 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0173599-b2fd-486b-a2d5-47ab17bcc293 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:29:53.087732 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0173599-b2fd-486b-a2d5-47ab17bcc293 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:29:53.094014 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 621/1246] 10.222.0.81 () {56 vars in 1374 bytes} [Wed Jul 20 22:29:52 2022] GET /identity/v3/role_assignments?user.id=d5500cebdd9d4f8e9aecb45d6c955878&scope.project.id=385edf2e59e546d090ace36600e29032&role.id=9b4a83e2eca44a8da9f356bc40f8788a => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:29:55.208905 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0109307f-deb2-4332-a95e-1bb104326cf0 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:29:55.209327 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0109307f-deb2-4332-a95e-1bb104326cf0 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:29:55.209466 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0109307f-deb2-4332-a95e-1bb104326cf0 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:29:55.210835 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 626/1247] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:29:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:29:55.216273 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedab798-31c7-4c31-b246-f5c3745bca76 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:29:55.217220 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedab798-31c7-4c31-b246-f5c3745bca76 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:29:55.217360 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedab798-31c7-4c31-b246-f5c3745bca76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:29:55.229596 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:29:55.241717 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-cedab798-31c7-4c31-b246-f5c3745bca76 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:29:55.242949 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-cedab798-31c7-4c31-b246-f5c3745bca76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:29:55.309324 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 622/1248] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:29:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 22:29:55.333494 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0109307f-deb2-4332-a95e-1bb104326cf0 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:29:55.353994 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:29:55.434932 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0109307f-deb2-4332-a95e-1bb104326cf0 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:29:55.468516 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af7df16-cc3b-4499-b2c3-73696606eeaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:29:55.469245 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af7df16-cc3b-4499-b2c3-73696606eeaf None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:29:55.469534 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af7df16-cc3b-4499-b2c3-73696606eeaf None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:29:55.469815 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af7df16-cc3b-4499-b2c3-73696606eeaf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:29:55.479596 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af7df16-cc3b-4499-b2c3-73696606eeaf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 22:29:55.487701 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-0af7df16-cc3b-4499-b2c3-73696606eeaf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 22:29:55.487701 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:29:55.487701 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:29:55.487701 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:29:55.487701 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:29:55.487701 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:29:55.487701 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:29:55.487701 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:29:55.487701 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:29:55.487701 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:29:55.487701 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:29:55.487701 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:29:55.487701 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 22:29:55.487701 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 22:29:55.487701 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 22:29:55.487701 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 22:29:55.494109 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 22:29:55.494109 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 22:29:55.494109 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 22:29:55.494109 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 22:29:55.494109 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:29:55.494109 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:29:55.494109 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:29:55.494109 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:29:55.494109 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:29:55.494109 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:29:55.494109 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:29:55.494109 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:29:55.494109 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:29:55.494109 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:29:55.494109 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:29:55.494109 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:29:55.494996 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:29:55.494996 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:29:55.494996 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:29:55.494996 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:29:55.494996 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 22:29:55.494996 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 22:29:55.494996 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 22:29:55.494996 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 22:29:55.494996 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 22:29:55.494996 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 22:29:55.494996 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 22:29:55.494996 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:29:55.494996 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 627/1249] 10.222.0.81 () {56 vars in 1096 bytes} [Wed Jul 20 22:29:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:29:55.495789 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedab798-31c7-4c31-b246-f5c3745bca76 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:29:55.593612 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedab798-31c7-4c31-b246-f5c3745bca76 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:29:55.622498 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:29:55.627269 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4769f1c-981a-46b5-8ebb-887b96521a2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:29:55.627942 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4769f1c-981a-46b5-8ebb-887b96521a2a None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:29:55.628228 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4769f1c-981a-46b5-8ebb-887b96521a2a None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:29:55.628507 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4769f1c-981a-46b5-8ebb-887b96521a2a None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:29:55.629243 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4769f1c-981a-46b5-8ebb-887b96521a2a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:29:55.629553 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4769f1c-981a-46b5-8ebb-887b96521a2a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:29:55.630911 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4769f1c-981a-46b5-8ebb-887b96521a2a None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:29:55.638733 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 623/1250] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Jul 20 22:29:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:29:55.644556 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af7df16-cc3b-4499-b2c3-73696606eeaf None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:29:55.720532 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:29:55.742940 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af7df16-cc3b-4499-b2c3-73696606eeaf None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:29:55.776841 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efec9243-1906-4816-a2f3-65bd9c7c7570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:29:55.777633 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efec9243-1906-4816-a2f3-65bd9c7c7570 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:29:55.777919 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efec9243-1906-4816-a2f3-65bd9c7c7570 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:29:55.778200 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efec9243-1906-4816-a2f3-65bd9c7c7570 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:29:55.788799 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efec9243-1906-4816-a2f3-65bd9c7c7570 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:29:55.794209 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efec9243-1906-4816-a2f3-65bd9c7c7570 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:29:55.808565 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-efec9243-1906-4816-a2f3-65bd9c7c7570 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:29:55.808565 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:29:55.808565 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:29:55.808565 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:29:55.808565 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:29:55.808565 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:29:55.808565 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:29:55.808565 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:29:55.808565 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:29:55.808565 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:29:55.808565 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:29:55.808565 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:29:55.808565 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 22:29:55.808565 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 22:29:55.808565 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 22:29:55.809609 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 22:29:55.809609 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:29:55.809609 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:29:55.809609 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:29:55.809609 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:29:55.809609 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:29:55.809609 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:29:55.809609 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:29:55.809609 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:29:55.809609 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:29:55.809609 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:29:55.809609 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:29:55.809609 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:29:55.809609 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:29:55.809609 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:29:55.809609 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:29:55.809609 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:29:55.815030 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 22:29:55.815030 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 22:29:55.815030 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:29:55.815030 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:29:55.815030 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 628/1251] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Jul 20 22:29:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:29:55.816120 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4769f1c-981a-46b5-8ebb-887b96521a2a None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:29:55.913075 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4769f1c-981a-46b5-8ebb-887b96521a2a None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:29:55.946218 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb042d9-b4d0-47b3-8988-d8ad50968f2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:29:55.946879 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb042d9-b4d0-47b3-8988-d8ad50968f2b None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:29:55.947193 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb042d9-b4d0-47b3-8988-d8ad50968f2b None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:29:55.947473 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb042d9-b4d0-47b3-8988-d8ad50968f2b None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:29:55.948181 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb042d9-b4d0-47b3-8988-d8ad50968f2b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:29:55.948488 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb042d9-b4d0-47b3-8988-d8ad50968f2b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:29:55.949853 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb042d9-b4d0-47b3-8988-d8ad50968f2b None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:29:55.961495 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 624/1252] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Jul 20 22:29:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:29:55.967178 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efec9243-1906-4816-a2f3-65bd9c7c7570 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:29:56.066327 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efec9243-1906-4816-a2f3-65bd9c7c7570 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:29:56.091832 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:29:56.099461 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e742eb39-3b49-48de-b5ae-9339e6e28c38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:29:56.100267 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e742eb39-3b49-48de-b5ae-9339e6e28c38 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:29:56.100555 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e742eb39-3b49-48de-b5ae-9339e6e28c38 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:29:56.100837 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e742eb39-3b49-48de-b5ae-9339e6e28c38 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:29:56.108583 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e742eb39-3b49-48de-b5ae-9339e6e28c38 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:29:56.110024 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e742eb39-3b49-48de-b5ae-9339e6e28c38 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:29:56.125045 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-e742eb39-3b49-48de-b5ae-9339e6e28c38 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 20 22:29:56.125045 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:29:56.125045 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:29:56.125045 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:29:56.125045 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:29:56.125045 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:29:56.125045 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:29:56.125045 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:29:56.125045 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:29:56.125045 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:29:56.125045 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:29:56.125045 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:29:56.125045 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 22:29:56.125045 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 22:29:56.125045 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 22:29:56.125045 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 22:29:56.126148 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:29:56.126148 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:29:56.126148 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 22:29:56.126148 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 22:29:56.126148 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 22:29:56.126148 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 22:29:56.126148 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:29:56.126148 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:29:56.126148 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:29:56.126148 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:29:56.126148 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:29:56.126148 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:29:56.126148 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:29:56.126148 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:29:56.126148 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:29:56.126148 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:29:56.126148 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:29:56.126148 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:29:56.132278 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:29:56.132278 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:29:56.132278 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 22:29:56.132278 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 22:29:56.132278 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 22:29:56.132278 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 22:29:56.132278 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 22:29:56.132278 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 22:29:56.132278 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 20 22:29:56.132278 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:29:56.132278 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 629/1253] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Jul 20 22:29:55 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:29:56.132867 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb042d9-b4d0-47b3-8988-d8ad50968f2b None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:29:56.188085 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:29:56.231522 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb042d9-b4d0-47b3-8988-d8ad50968f2b None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:29:56.265277 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1782e5de-22a7-4cab-84f1-e068e007797f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:29:56.265949 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1782e5de-22a7-4cab-84f1-e068e007797f None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:29:56.266231 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1782e5de-22a7-4cab-84f1-e068e007797f None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:29:56.266504 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1782e5de-22a7-4cab-84f1-e068e007797f None admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:29:56.267469 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1782e5de-22a7-4cab-84f1-e068e007797f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:29:56.272680 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1782e5de-22a7-4cab-84f1-e068e007797f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:29:56.272680 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1782e5de-22a7-4cab-84f1-e068e007797f None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:29:56.287116 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 625/1254] 10.222.0.81 () {56 vars in 1143 bytes} [Wed Jul 20 22:29:56 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:29:56.292818 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e742eb39-3b49-48de-b5ae-9339e6e28c38 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:29:56.391140 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e742eb39-3b49-48de-b5ae-9339e6e28c38 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:29:56.426579 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae42d6e-33d0-4dbc-8ee4-97eeec8e38ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:29:56.427253 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae42d6e-33d0-4dbc-8ee4-97eeec8e38ed None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:29:56.427544 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae42d6e-33d0-4dbc-8ee4-97eeec8e38ed None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:29:56.427844 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae42d6e-33d0-4dbc-8ee4-97eeec8e38ed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:29:56.480822 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:29:56.493295 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae42d6e-33d0-4dbc-8ee4-97eeec8e38ed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:29:56.494744 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae42d6e-33d0-4dbc-8ee4-97eeec8e38ed None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:29:56.590167 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 630/1255] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Jul 20 22:29:56 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 22:29:56.596539 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1782e5de-22a7-4cab-84f1-e068e007797f None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:29:56.697667 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1782e5de-22a7-4cab-84f1-e068e007797f None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:29:56.710941 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:29:56.731882 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23494d2-54b8-4d70-b782-a9507fdb1598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:29:56.736283 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23494d2-54b8-4d70-b782-a9507fdb1598 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:29:56.736283 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23494d2-54b8-4d70-b782-a9507fdb1598 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:29:56.736283 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23494d2-54b8-4d70-b782-a9507fdb1598 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:29:56.743305 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23494d2-54b8-4d70-b782-a9507fdb1598 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:29:56.744777 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23494d2-54b8-4d70-b782-a9507fdb1598 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:29:56.762301 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-d23494d2-54b8-4d70-b782-a9507fdb1598 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:29:56.762301 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:29:56.762301 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:29:56.762301 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:29:56.762301 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:29:56.762301 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:29:56.762301 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:29:56.762301 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:29:56.762301 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:29:56.762301 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:29:56.762301 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:29:56.762301 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:29:56.762301 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 22:29:56.762301 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 22:29:56.762301 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 22:29:56.763350 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 22:29:56.763350 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:29:56.763350 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:29:56.763350 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:29:56.763350 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:29:56.763350 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:29:56.763350 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:29:56.763350 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:29:56.763350 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:29:56.763350 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:29:56.763350 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:29:56.763350 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:29:56.763350 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:29:56.763350 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:29:56.763350 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:29:56.763350 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:29:56.763350 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:29:56.764967 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 22:29:56.764967 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 22:29:56.764967 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:29:56.764967 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:29:56.764967 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 626/1256] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Jul 20 22:29:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:29:56.769850 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae42d6e-33d0-4dbc-8ee4-97eeec8e38ed None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:29:56.855281 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:29:56.867072 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae42d6e-33d0-4dbc-8ee4-97eeec8e38ed None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:29:56.901715 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a50326-a1f2-47e4-85c1-2938ebbab6c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:29:56.902379 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a50326-a1f2-47e4-85c1-2938ebbab6c6 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:29:56.902671 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a50326-a1f2-47e4-85c1-2938ebbab6c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:29:56.902953 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a50326-a1f2-47e4-85c1-2938ebbab6c6 None admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:29:56.903729 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a50326-a1f2-47e4-85c1-2938ebbab6c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:29:56.904042 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a50326-a1f2-47e4-85c1-2938ebbab6c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:29:56.905438 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a50326-a1f2-47e4-85c1-2938ebbab6c6 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:29:56.916572 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 631/1257] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Jul 20 22:29:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:29:56.922464 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23494d2-54b8-4d70-b782-a9507fdb1598 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:29:57.020908 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23494d2-54b8-4d70-b782-a9507fdb1598 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:29:57.054936 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651d35cd-505a-42f6-affb-0e6d8129c6f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:29:57.055766 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651d35cd-505a-42f6-affb-0e6d8129c6f8 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:29:57.056121 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651d35cd-505a-42f6-affb-0e6d8129c6f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:29:57.056459 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651d35cd-505a-42f6-affb-0e6d8129c6f8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:29:57.063687 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651d35cd-505a-42f6-affb-0e6d8129c6f8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:29:57.065157 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651d35cd-505a-42f6-affb-0e6d8129c6f8 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:29:57.079739 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-651d35cd-505a-42f6-affb-0e6d8129c6f8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 22:29:57.079739 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:29:57.079739 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:29:57.079739 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:29:57.079739 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:29:57.079739 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:29:57.079739 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:29:57.079739 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:29:57.079739 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:29:57.079739 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:29:57.079739 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:29:57.079739 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:29:57.079739 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 22:29:57.079739 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 22:29:57.079739 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 22:29:57.083447 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 22:29:57.083447 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:29:57.083447 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:29:57.083447 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:29:57.083447 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:29:57.083447 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:29:57.083447 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:29:57.083447 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:29:57.083447 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:29:57.083447 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:29:57.083447 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:29:57.083447 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:29:57.083447 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:29:57.083447 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:29:57.083447 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:29:57.083447 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:29:57.083447 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:29:57.084543 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 22:29:57.084543 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 22:29:57.084543 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 22:29:57.084543 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 22:29:57.084543 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 22:29:57.084543 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 22:29:57.084543 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 22:29:57.084543 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:29:57.084543 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 627/1258] 10.222.0.81 () {56 vars in 1111 bytes} [Wed Jul 20 22:29:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:29:57.086983 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a50326-a1f2-47e4-85c1-2938ebbab6c6 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:29:57.187277 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a50326-a1f2-47e4-85c1-2938ebbab6c6 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:29:57.222493 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ec42be-32a5-4fee-9eb4-04d8d5295848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:29:57.223193 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ec42be-32a5-4fee-9eb4-04d8d5295848 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:29:57.223478 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ec42be-32a5-4fee-9eb4-04d8d5295848 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:29:57.223761 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ec42be-32a5-4fee-9eb4-04d8d5295848 None admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:29:57.224547 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ec42be-32a5-4fee-9eb4-04d8d5295848 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:29:57.224861 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ec42be-32a5-4fee-9eb4-04d8d5295848 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:29:57.226288 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ec42be-32a5-4fee-9eb4-04d8d5295848 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:29:57.238205 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 632/1259] 10.222.0.81 () {56 vars in 1148 bytes} [Wed Jul 20 22:29:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:29:57.244459 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651d35cd-505a-42f6-affb-0e6d8129c6f8 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:29:57.270864 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:29:57.341680 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651d35cd-505a-42f6-affb-0e6d8129c6f8 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:29:57.376354 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74abbd1-66de-4632-b1ad-4990ad869eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:29:57.377021 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74abbd1-66de-4632-b1ad-4990ad869eb9 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:29:57.377299 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74abbd1-66de-4632-b1ad-4990ad869eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:29:57.377604 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74abbd1-66de-4632-b1ad-4990ad869eb9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:29:57.378426 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74abbd1-66de-4632-b1ad-4990ad869eb9 None admin] RBAC: Adding query filter params (role.id=d89b6c92d35a401a8517f1d47f6be843, scope.project.id=385edf2e59e546d090ace36600e29032, user.id=d5500cebdd9d4f8e9aecb45d6c955878) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:29:57.378736 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74abbd1-66de-4632-b1ad-4990ad869eb9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:29:57.380202 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74abbd1-66de-4632-b1ad-4990ad869eb9 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:29:57.385634 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 628/1260] 10.222.0.81 () {56 vars in 1374 bytes} [Wed Jul 20 22:29:57 2022] GET /identity/v3/role_assignments?user.id=d5500cebdd9d4f8e9aecb45d6c955878&scope.project.id=385edf2e59e546d090ace36600e29032&role.id=d89b6c92d35a401a8517f1d47f6be843 => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:29:59.390636 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9f5acc-fe16-4ff6-af5d-db94dee9192a None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:29:59.391211 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9f5acc-fe16-4ff6-af5d-db94dee9192a None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:29:59.391318 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9f5acc-fe16-4ff6-af5d-db94dee9192a None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:29:59.392636 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 633/1261] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:29:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:29:59.397873 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846ec623-6dca-4649-b641-00b6c02273b8 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:29:59.404099 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846ec623-6dca-4649-b641-00b6c02273b8 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:29:59.404099 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846ec623-6dca-4649-b641-00b6c02273b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:29:59.414694 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-846ec623-6dca-4649-b641-00b6c02273b8 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:29:59.415986 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-846ec623-6dca-4649-b641-00b6c02273b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:29:59.482024 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 629/1262] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:29:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 22:29:59.505309 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9f5acc-fe16-4ff6-af5d-db94dee9192a None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:29:59.516704 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:29:59.605624 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9f5acc-fe16-4ff6-af5d-db94dee9192a None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:29:59.641915 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d4552a-465b-4f3a-8d41-39fd562a5773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:29:59.642720 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d4552a-465b-4f3a-8d41-39fd562a5773 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:29:59.643031 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d4552a-465b-4f3a-8d41-39fd562a5773 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:29:59.643384 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d4552a-465b-4f3a-8d41-39fd562a5773 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:29:59.653799 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d4552a-465b-4f3a-8d41-39fd562a5773 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 22:29:59.661982 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-f0d4552a-465b-4f3a-8d41-39fd562a5773 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 22:29:59.661982 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:29:59.661982 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:29:59.661982 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:29:59.661982 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:29:59.661982 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:29:59.661982 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:29:59.661982 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:29:59.661982 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:29:59.661982 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:29:59.661982 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:29:59.661982 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:29:59.661982 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 22:29:59.661982 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 22:29:59.661982 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 22:29:59.661982 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 22:29:59.666766 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 22:29:59.666766 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 22:29:59.666766 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 22:29:59.666766 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 22:29:59.666766 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:29:59.666766 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:29:59.666766 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:29:59.666766 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:29:59.666766 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:29:59.666766 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:29:59.666766 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:29:59.666766 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:29:59.666766 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:29:59.666766 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:29:59.666766 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:29:59.666766 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:29:59.667653 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:29:59.667653 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:29:59.667653 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:29:59.667653 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:29:59.667653 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 22:29:59.667653 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 22:29:59.667653 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 22:29:59.667653 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 22:29:59.667653 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 22:29:59.667653 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 22:29:59.667653 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 22:29:59.667653 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:29:59.667653 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 634/1263] 10.222.0.81 () {56 vars in 1096 bytes} [Wed Jul 20 22:29:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:29:59.669266 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846ec623-6dca-4649-b641-00b6c02273b8 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:29:59.689463 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:29:59.768548 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846ec623-6dca-4649-b641-00b6c02273b8 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:29:59.801504 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c1c0b7-6a2f-4168-80da-8aaf399e5b3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:29:59.802182 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c1c0b7-6a2f-4168-80da-8aaf399e5b3b None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:29:59.802470 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c1c0b7-6a2f-4168-80da-8aaf399e5b3b None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:29:59.802753 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c1c0b7-6a2f-4168-80da-8aaf399e5b3b None admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:29:59.803527 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c1c0b7-6a2f-4168-80da-8aaf399e5b3b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:29:59.803841 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c1c0b7-6a2f-4168-80da-8aaf399e5b3b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:29:59.805207 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c1c0b7-6a2f-4168-80da-8aaf399e5b3b None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:29:59.814156 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 630/1264] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Jul 20 22:29:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:29:59.820148 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d4552a-465b-4f3a-8d41-39fd562a5773 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:29:59.886392 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:29:59.918819 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d4552a-465b-4f3a-8d41-39fd562a5773 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:29:59.951773 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3794e43-673d-4847-b870-0fbb2ef5b60a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:29:59.952580 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3794e43-673d-4847-b870-0fbb2ef5b60a None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:29:59.952868 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3794e43-673d-4847-b870-0fbb2ef5b60a None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:29:59.953193 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3794e43-673d-4847-b870-0fbb2ef5b60a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:29:59.963391 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3794e43-673d-4847-b870-0fbb2ef5b60a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:29:59.964850 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3794e43-673d-4847-b870-0fbb2ef5b60a None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:29:59.982889 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-c3794e43-673d-4847-b870-0fbb2ef5b60a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:29:59.982889 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:29:59.982889 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:29:59.982889 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:29:59.982889 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:29:59.982889 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:29:59.982889 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:29:59.982889 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:29:59.982889 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:29:59.982889 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:29:59.982889 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:29:59.982889 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:29:59.982889 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 22:29:59.982889 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 22:29:59.982889 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 22:29:59.984556 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 22:29:59.984556 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:29:59.984556 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:29:59.984556 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:29:59.984556 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:29:59.984556 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:29:59.984556 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:29:59.984556 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:29:59.984556 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:29:59.984556 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:29:59.984556 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:29:59.984556 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:29:59.984556 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:29:59.984556 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:29:59.984556 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:29:59.984556 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:29:59.984556 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:29:59.985667 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 22:29:59.985667 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 22:29:59.985667 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:29:59.985667 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:29:59.985667 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 635/1265] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Jul 20 22:29:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:29:59.990754 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c1c0b7-6a2f-4168-80da-8aaf399e5b3b None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:30:00.068945 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:30:00.087128 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c1c0b7-6a2f-4168-80da-8aaf399e5b3b None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:30:00.120302 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb4fc31-153b-4e3d-ad56-045579768c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:30:00.120982 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb4fc31-153b-4e3d-ad56-045579768c7a None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:30:00.121268 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb4fc31-153b-4e3d-ad56-045579768c7a None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:30:00.121564 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb4fc31-153b-4e3d-ad56-045579768c7a None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:30:00.122265 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb4fc31-153b-4e3d-ad56-045579768c7a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:30:00.122569 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb4fc31-153b-4e3d-ad56-045579768c7a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:30:00.124462 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb4fc31-153b-4e3d-ad56-045579768c7a None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:30:00.135600 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 631/1266] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Jul 20 22:29:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:30:00.142277 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3794e43-673d-4847-b870-0fbb2ef5b60a None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:30:00.244752 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3794e43-673d-4847-b870-0fbb2ef5b60a None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:30:00.265625 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:30:00.283297 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6e34fa-555b-409e-8af2-2f340e0dd576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:30:00.284117 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6e34fa-555b-409e-8af2-2f340e0dd576 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:30:00.284403 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6e34fa-555b-409e-8af2-2f340e0dd576 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:30:00.284676 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6e34fa-555b-409e-8af2-2f340e0dd576 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:30:00.292205 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6e34fa-555b-409e-8af2-2f340e0dd576 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:30:00.293640 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6e34fa-555b-409e-8af2-2f340e0dd576 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:30:00.308953 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-7a6e34fa-555b-409e-8af2-2f340e0dd576 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 20 22:30:00.308953 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:30:00.308953 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:30:00.308953 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:30:00.308953 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:30:00.308953 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:30:00.308953 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:30:00.308953 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:30:00.308953 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:30:00.308953 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:30:00.308953 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:30:00.308953 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:30:00.308953 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 22:30:00.308953 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 22:30:00.308953 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 22:30:00.308953 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 22:30:00.310093 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:30:00.310093 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:30:00.310093 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 22:30:00.310093 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 22:30:00.310093 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 22:30:00.310093 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 22:30:00.310093 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:30:00.310093 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:30:00.310093 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:30:00.310093 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:30:00.310093 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:30:00.310093 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:30:00.310093 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:30:00.310093 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:30:00.310093 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:30:00.310093 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:30:00.310093 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:30:00.310093 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:30:00.311119 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:30:00.311119 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:30:00.311119 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 22:30:00.311119 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 22:30:00.311119 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 22:30:00.311119 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 22:30:00.311119 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 22:30:00.311119 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 22:30:00.311119 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 20 22:30:00.311119 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:30:00.311119 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 636/1267] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Jul 20 22:30:00 2022] GET /identity/v3/users/placement => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:30:00.316626 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb4fc31-153b-4e3d-ad56-045579768c7a None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:30:00.413090 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb4fc31-153b-4e3d-ad56-045579768c7a None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:30:00.446493 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadaaeee-4791-4bad-b4e6-d709c5710205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:30:00.447207 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadaaeee-4791-4bad-b4e6-d709c5710205 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:30:00.447491 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadaaeee-4791-4bad-b4e6-d709c5710205 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:30:00.447766 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadaaeee-4791-4bad-b4e6-d709c5710205 None admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:30:00.448589 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadaaeee-4791-4bad-b4e6-d709c5710205 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:30:00.448905 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadaaeee-4791-4bad-b4e6-d709c5710205 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:30:00.450316 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadaaeee-4791-4bad-b4e6-d709c5710205 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:30:00.459800 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:30:00.468489 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 632/1268] 10.222.0.81 () {56 vars in 1143 bytes} [Wed Jul 20 22:30:00 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:30:00.474142 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6e34fa-555b-409e-8af2-2f340e0dd576 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:30:00.572770 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6e34fa-555b-409e-8af2-2f340e0dd576 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:30:00.608954 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2eb85d8-ad72-4ce5-8238-2bbf406d797d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:30:00.609824 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eb85d8-ad72-4ce5-8238-2bbf406d797d None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:30:00.610107 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eb85d8-ad72-4ce5-8238-2bbf406d797d None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:30:00.610380 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eb85d8-ad72-4ce5-8238-2bbf406d797d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:30:00.621021 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2eb85d8-ad72-4ce5-8238-2bbf406d797d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:30:00.622710 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2eb85d8-ad72-4ce5-8238-2bbf406d797d None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:30:00.643806 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-d2eb85d8-ad72-4ce5-8238-2bbf406d797d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:30:00.643806 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:30:00.643806 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:30:00.643806 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:30:00.643806 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:30:00.643806 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:30:00.643806 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:30:00.643806 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:30:00.643806 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:30:00.643806 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:30:00.643806 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:30:00.643806 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:30:00.643806 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 22:30:00.643806 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 22:30:00.643806 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 22:30:00.644923 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 22:30:00.644923 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:30:00.644923 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:30:00.644923 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:30:00.644923 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:30:00.644923 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:30:00.644923 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:30:00.644923 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:30:00.644923 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:30:00.644923 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:30:00.644923 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:30:00.644923 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:30:00.644923 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:30:00.644923 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:30:00.644923 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:30:00.644923 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:30:00.644923 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:30:00.646038 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 22:30:00.646038 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 22:30:00.646038 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:30:00.646038 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:30:00.646038 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 637/1269] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Jul 20 22:30:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:30:00.652245 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadaaeee-4791-4bad-b4e6-d709c5710205 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:30:00.754709 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadaaeee-4791-4bad-b4e6-d709c5710205 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:30:00.789915 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8625d56f-9463-4e00-be63-c87f3a1d045f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:30:00.790584 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8625d56f-9463-4e00-be63-c87f3a1d045f None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:30:00.790874 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8625d56f-9463-4e00-be63-c87f3a1d045f None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:30:00.791198 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8625d56f-9463-4e00-be63-c87f3a1d045f None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:30:00.791917 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8625d56f-9463-4e00-be63-c87f3a1d045f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:30:00.792227 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8625d56f-9463-4e00-be63-c87f3a1d045f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:30:00.793605 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8625d56f-9463-4e00-be63-c87f3a1d045f None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:30:00.804772 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 633/1270] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Jul 20 22:30:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 22:30:00.812064 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2eb85d8-ad72-4ce5-8238-2bbf406d797d None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:30:00.843224 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:30:00.909584 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2eb85d8-ad72-4ce5-8238-2bbf406d797d None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:30:00.945428 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88705b76-1f2c-435d-95a1-03bfffc8c2af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:30:00.947981 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88705b76-1f2c-435d-95a1-03bfffc8c2af None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:30:00.947981 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88705b76-1f2c-435d-95a1-03bfffc8c2af None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:30:00.947981 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88705b76-1f2c-435d-95a1-03bfffc8c2af None admin] PATH_INFO: `/v3/projects/service` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:30:00.955274 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88705b76-1f2c-435d-95a1-03bfffc8c2af None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:30:00.956821 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88705b76-1f2c-435d-95a1-03bfffc8c2af None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:30:00.972136 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-88705b76-1f2c-435d-95a1-03bfffc8c2af None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 22:30:00.972136 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:30:00.972136 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:30:00.972136 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:30:00.972136 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:30:00.972136 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:30:00.972136 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:30:00.972136 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:30:00.972136 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:30:00.972136 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:30:00.972136 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:30:00.972136 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:30:00.972136 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 22:30:00.972136 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 22:30:00.972136 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 22:30:00.973476 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 22:30:00.973476 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:30:00.973476 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:30:00.973476 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:30:00.973476 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:30:00.973476 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:30:00.973476 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:30:00.973476 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:30:00.973476 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:30:00.973476 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:30:00.973476 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:30:00.973476 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:30:00.973476 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:30:00.973476 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:30:00.973476 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:30:00.973476 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:30:00.973476 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:30:00.974808 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 22:30:00.974808 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 22:30:00.974808 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 22:30:00.974808 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 22:30:00.974808 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 22:30:00.974808 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 22:30:00.974808 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 22:30:00.974808 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:30:00.974808 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 638/1271] 10.222.0.81 () {56 vars in 1111 bytes} [Wed Jul 20 22:30:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:30:00.979348 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8625d56f-9463-4e00-be63-c87f3a1d045f None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:30:01.038652 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:30:01.078040 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8625d56f-9463-4e00-be63-c87f3a1d045f None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:30:01.109898 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195ecde0-480c-4e5e-b145-0336f9b08fc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:30:01.110573 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195ecde0-480c-4e5e-b145-0336f9b08fc8 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:30:01.110858 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195ecde0-480c-4e5e-b145-0336f9b08fc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:30:01.111168 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195ecde0-480c-4e5e-b145-0336f9b08fc8 None admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:30:01.111943 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195ecde0-480c-4e5e-b145-0336f9b08fc8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:30:01.112251 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195ecde0-480c-4e5e-b145-0336f9b08fc8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:30:01.113669 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195ecde0-480c-4e5e-b145-0336f9b08fc8 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:30:01.126092 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 634/1272] 10.222.0.81 () {56 vars in 1148 bytes} [Wed Jul 20 22:30:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:30:01.132275 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88705b76-1f2c-435d-95a1-03bfffc8c2af None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:30:01.224333 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:30:01.232660 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88705b76-1f2c-435d-95a1-03bfffc8c2af None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:30:01.267901 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44bb29f-c5c9-4ba6-ae3d-2ed70bd7cd89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:30:01.268818 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44bb29f-c5c9-4ba6-ae3d-2ed70bd7cd89 None admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:30:01.269106 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44bb29f-c5c9-4ba6-ae3d-2ed70bd7cd89 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:30:01.269387 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44bb29f-c5c9-4ba6-ae3d-2ed70bd7cd89 None admin] PATH_INFO: `/v3/projects/385edf2e59e546d090ace36600e29032/users/d5500cebdd9d4f8e9aecb45d6c955878/roles/d89b6c92d35a401a8517f1d47f6be843` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:30:01.272349 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44bb29f-c5c9-4ba6-ae3d-2ed70bd7cd89 None admin] RBAC: Authorizing `identity:create_grant(project_id=385edf2e59e546d090ace36600e29032, user_id=d5500cebdd9d4f8e9aecb45d6c955878, role_id=d89b6c92d35a401a8517f1d47f6be843)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:30:01.274983 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44bb29f-c5c9-4ba6-ae3d-2ed70bd7cd89 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:30:01.367493 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 639/1273] 10.222.0.81 () {58 vars in 1439 bytes} [Wed Jul 20 22:30:01 2022] PUT /identity/v3/projects/385edf2e59e546d090ace36600e29032/users/d5500cebdd9d4f8e9aecb45d6c955878/roles/d89b6c92d35a401a8517f1d47f6be843 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:30:03.388722 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbc4e4a-1c7c-404e-a524-23392cf98c22 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:30:03.389276 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbc4e4a-1c7c-404e-a524-23392cf98c22 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:30:03.389276 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbc4e4a-1c7c-404e-a524-23392cf98c22 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:30:03.390650 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 635/1274] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:30:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:30:03.395902 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882001a9-ee63-4f2d-9610-b8fcbd040d84 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:30:03.396322 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882001a9-ee63-4f2d-9610-b8fcbd040d84 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:30:03.399923 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882001a9-ee63-4f2d-9610-b8fcbd040d84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:30:03.412885 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-882001a9-ee63-4f2d-9610-b8fcbd040d84 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:30:03.414063 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-882001a9-ee63-4f2d-9610-b8fcbd040d84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:30:03.480648 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 640/1275] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:30:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 22:30:03.505270 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbc4e4a-1c7c-404e-a524-23392cf98c22 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:30:03.605111 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbc4e4a-1c7c-404e-a524-23392cf98c22 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:30:03.622832 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:30:03.639606 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f344d0-7158-4317-8f8d-4683c169234b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:30:03.640459 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f344d0-7158-4317-8f8d-4683c169234b None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:30:03.640873 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f344d0-7158-4317-8f8d-4683c169234b None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:30:03.641148 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f344d0-7158-4317-8f8d-4683c169234b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:30:03.650802 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f344d0-7158-4317-8f8d-4683c169234b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 22:30:03.660230 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-a2f344d0-7158-4317-8f8d-4683c169234b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 22:30:03.660230 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:30:03.660230 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:30:03.660230 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:30:03.660230 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:30:03.660230 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:30:03.660230 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:30:03.660230 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:30:03.660230 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:30:03.660230 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:30:03.660230 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:30:03.660230 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:30:03.660230 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 22:30:03.660230 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 22:30:03.660230 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 22:30:03.660230 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 22:30:03.664683 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 22:30:03.664683 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 22:30:03.664683 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 22:30:03.664683 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 22:30:03.664683 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:30:03.664683 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:30:03.664683 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:30:03.664683 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:30:03.664683 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:30:03.664683 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:30:03.664683 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:30:03.664683 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:30:03.664683 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:30:03.664683 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:30:03.664683 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:30:03.664683 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:30:03.665757 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:30:03.665757 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:30:03.665757 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:30:03.665757 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:30:03.665757 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 22:30:03.665757 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 22:30:03.665757 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 22:30:03.665757 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 22:30:03.665757 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 22:30:03.665757 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 22:30:03.665757 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 22:30:03.665757 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:30:03.665757 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 636/1276] 10.222.0.81 () {56 vars in 1096 bytes} [Wed Jul 20 22:30:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:30:03.667779 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882001a9-ee63-4f2d-9610-b8fcbd040d84 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:30:03.749354 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:30:03.767860 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882001a9-ee63-4f2d-9610-b8fcbd040d84 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:30:03.800797 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961d0c7b-387c-4c5c-937a-61acda98f485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:30:03.801471 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961d0c7b-387c-4c5c-937a-61acda98f485 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:30:03.801754 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961d0c7b-387c-4c5c-937a-61acda98f485 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:30:03.802031 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961d0c7b-387c-4c5c-937a-61acda98f485 None admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:30:03.802806 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961d0c7b-387c-4c5c-937a-61acda98f485 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:30:03.803146 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961d0c7b-387c-4c5c-937a-61acda98f485 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:30:03.804493 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961d0c7b-387c-4c5c-937a-61acda98f485 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:30:03.813035 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 641/1277] 10.222.0.81 () {56 vars in 1099 bytes} [Wed Jul 20 22:30:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:30:03.818862 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f344d0-7158-4317-8f8d-4683c169234b None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:30:03.919674 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f344d0-7158-4317-8f8d-4683c169234b None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:30:03.952292 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1fad70-23e4-40ce-ae22-b2b47e95d9bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:30:03.953703 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1fad70-23e4-40ce-ae22-b2b47e95d9bc None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:30:03.953991 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1fad70-23e4-40ce-ae22-b2b47e95d9bc None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:30:03.954270 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1fad70-23e4-40ce-ae22-b2b47e95d9bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:30:03.964617 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1fad70-23e4-40ce-ae22-b2b47e95d9bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:30:03.966055 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1fad70-23e4-40ce-ae22-b2b47e95d9bc None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:30:03.983826 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-7e1fad70-23e4-40ce-ae22-b2b47e95d9bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:30:03.983826 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:30:03.983826 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:30:03.983826 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:30:03.983826 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:30:03.983826 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:30:03.983826 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:30:03.983826 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:30:03.983826 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:30:03.983826 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:30:03.983826 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:30:03.983826 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:30:03.983826 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 22:30:03.983826 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 22:30:03.983826 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 22:30:03.986586 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 22:30:03.986586 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:30:03.986586 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:30:03.986586 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:30:03.986586 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:30:03.986586 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:30:03.986586 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:30:03.986586 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:30:03.986586 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:30:03.986586 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:30:03.986586 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:30:03.986586 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:30:03.986586 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:30:03.986586 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:30:03.986586 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:30:03.986586 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:30:03.986586 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:30:03.987812 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 22:30:03.987812 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 22:30:03.987812 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:30:03.987812 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:30:03.987812 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 637/1278] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Jul 20 22:30:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:30:03.991573 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961d0c7b-387c-4c5c-937a-61acda98f485 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:30:04.097299 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961d0c7b-387c-4c5c-937a-61acda98f485 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:30:04.129344 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457ff154-d241-4d2c-8fb2-dc25e2db1f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:30:04.138088 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457ff154-d241-4d2c-8fb2-dc25e2db1f0d None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:30:04.138088 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457ff154-d241-4d2c-8fb2-dc25e2db1f0d None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:30:04.138088 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457ff154-d241-4d2c-8fb2-dc25e2db1f0d None admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:30:04.138088 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457ff154-d241-4d2c-8fb2-dc25e2db1f0d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:30:04.138088 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457ff154-d241-4d2c-8fb2-dc25e2db1f0d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:30:04.138088 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457ff154-d241-4d2c-8fb2-dc25e2db1f0d None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:30:04.144768 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 642/1279] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Jul 20 22:30:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:30:04.150602 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1fad70-23e4-40ce-ae22-b2b47e95d9bc None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:30:04.181932 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:30:04.250263 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1fad70-23e4-40ce-ae22-b2b47e95d9bc None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:30:04.284473 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c15348-f1db-4cff-a8c7-4096110c6404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:30:04.285288 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c15348-f1db-4cff-a8c7-4096110c6404 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:30:04.285569 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c15348-f1db-4cff-a8c7-4096110c6404 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:30:04.292880 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c15348-f1db-4cff-a8c7-4096110c6404 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:30:04.293255 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c15348-f1db-4cff-a8c7-4096110c6404 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:30:04.294711 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c15348-f1db-4cff-a8c7-4096110c6404 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:30:04.309571 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-55c15348-f1db-4cff-a8c7-4096110c6404 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 20 22:30:04.309571 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:30:04.309571 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:30:04.309571 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:30:04.309571 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:30:04.309571 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:30:04.309571 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:30:04.309571 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:30:04.309571 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:30:04.309571 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:30:04.309571 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:30:04.309571 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:30:04.309571 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 22:30:04.309571 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 22:30:04.309571 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 22:30:04.309571 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 22:30:04.310511 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:30:04.310511 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:30:04.310511 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 22:30:04.310511 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 22:30:04.310511 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 22:30:04.310511 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 22:30:04.310511 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:30:04.310511 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:30:04.310511 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:30:04.310511 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:30:04.310511 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:30:04.310511 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:30:04.310511 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:30:04.310511 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:30:04.310511 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:30:04.310511 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:30:04.310511 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:30:04.310511 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:30:04.311635 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:30:04.311635 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:30:04.311635 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 22:30:04.311635 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 22:30:04.311635 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 22:30:04.311635 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 22:30:04.311635 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 22:30:04.311635 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 22:30:04.311635 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 20 22:30:04.311635 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:30:04.311635 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 638/1280] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Jul 20 22:30:04 2022] GET /identity/v3/users/placement => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:30:04.317363 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457ff154-d241-4d2c-8fb2-dc25e2db1f0d None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:30:04.321333 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:30:04.414018 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457ff154-d241-4d2c-8fb2-dc25e2db1f0d None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:30:04.446667 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37865ee6-dc0b-46e2-b1d5-ecdd7747f000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:30:04.447360 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37865ee6-dc0b-46e2-b1d5-ecdd7747f000 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:30:04.447648 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37865ee6-dc0b-46e2-b1d5-ecdd7747f000 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:30:04.447931 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37865ee6-dc0b-46e2-b1d5-ecdd7747f000 None admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:30:04.448759 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37865ee6-dc0b-46e2-b1d5-ecdd7747f000 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:30:04.449071 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37865ee6-dc0b-46e2-b1d5-ecdd7747f000 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:30:04.450485 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37865ee6-dc0b-46e2-b1d5-ecdd7747f000 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:30:04.468128 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 643/1281] 10.222.0.81 () {56 vars in 1143 bytes} [Wed Jul 20 22:30:04 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:30:04.474325 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c15348-f1db-4cff-a8c7-4096110c6404 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:30:04.565333 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:30:04.575232 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c15348-f1db-4cff-a8c7-4096110c6404 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:30:04.608889 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6752dd9-b15d-4d86-a1ce-529238da1081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:30:04.610150 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6752dd9-b15d-4d86-a1ce-529238da1081 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:30:04.619590 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6752dd9-b15d-4d86-a1ce-529238da1081 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:30:04.619590 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6752dd9-b15d-4d86-a1ce-529238da1081 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:30:04.679798 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6752dd9-b15d-4d86-a1ce-529238da1081 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:30:04.681223 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6752dd9-b15d-4d86-a1ce-529238da1081 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:30:04.772161 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:30:04.783048 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 639/1282] 10.222.0.81 () {58 vars in 1282 bytes} [Wed Jul 20 22:30:04 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 22:30:04.788865 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37865ee6-dc0b-46e2-b1d5-ecdd7747f000 None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:30:04.850545 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:30:04.888196 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37865ee6-dc0b-46e2-b1d5-ecdd7747f000 None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:30:04.921949 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51804da3-80b5-47d7-a69f-8ee853f565bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:30:04.922743 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51804da3-80b5-47d7-a69f-8ee853f565bc None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:30:04.923151 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51804da3-80b5-47d7-a69f-8ee853f565bc None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:30:04.923449 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51804da3-80b5-47d7-a69f-8ee853f565bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:30:04.934522 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51804da3-80b5-47d7-a69f-8ee853f565bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:30:04.936446 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51804da3-80b5-47d7-a69f-8ee853f565bc None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:30:04.954572 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-51804da3-80b5-47d7-a69f-8ee853f565bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:30:04.954572 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:30:04.954572 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:30:04.954572 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:30:04.954572 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:30:04.954572 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:30:04.954572 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:30:04.954572 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:30:04.954572 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:30:04.954572 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:30:04.954572 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:30:04.954572 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:30:04.954572 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 22:30:04.954572 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 22:30:04.954572 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 22:30:04.955682 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 22:30:04.955682 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:30:04.955682 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:30:04.955682 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:30:04.955682 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:30:04.955682 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:30:04.955682 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:30:04.955682 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:30:04.955682 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:30:04.955682 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:30:04.955682 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:30:04.955682 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:30:04.955682 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:30:04.955682 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:30:04.955682 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:30:04.955682 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:30:04.955682 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:30:04.956759 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 22:30:04.956759 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 22:30:04.956759 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 22:30:04.956759 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:30:04.956759 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 644/1283] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Jul 20 22:30:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:30:04.961778 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6752dd9-b15d-4d86-a1ce-529238da1081 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:30:05.060398 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6752dd9-b15d-4d86-a1ce-529238da1081 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:30:05.100026 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169dc0ab-e94a-4308-b500-92aad7a291a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:30:05.100690 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169dc0ab-e94a-4308-b500-92aad7a291a9 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:30:05.100972 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169dc0ab-e94a-4308-b500-92aad7a291a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:30:05.101246 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169dc0ab-e94a-4308-b500-92aad7a291a9 None admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:30:05.101950 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169dc0ab-e94a-4308-b500-92aad7a291a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:30:05.102252 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169dc0ab-e94a-4308-b500-92aad7a291a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:30:05.103626 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169dc0ab-e94a-4308-b500-92aad7a291a9 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:30:05.114986 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 640/1284] 10.222.0.81 () {56 vars in 1109 bytes} [Wed Jul 20 22:30:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:30:05.120820 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51804da3-80b5-47d7-a69f-8ee853f565bc None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:30:05.222415 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51804da3-80b5-47d7-a69f-8ee853f565bc None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:30:05.236120 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:30:05.255617 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581c24d3-0ae6-42ae-aac3-a7300ed26dba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:30:05.256416 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581c24d3-0ae6-42ae-aac3-a7300ed26dba None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:30:05.256701 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581c24d3-0ae6-42ae-aac3-a7300ed26dba None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:30:05.256980 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581c24d3-0ae6-42ae-aac3-a7300ed26dba None admin] PATH_INFO: `/v3/projects/service` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:30:05.263383 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581c24d3-0ae6-42ae-aac3-a7300ed26dba None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:30:05.269287 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581c24d3-0ae6-42ae-aac3-a7300ed26dba None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:30:05.279969 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application [None req-581c24d3-0ae6-42ae-aac3-a7300ed26dba None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 22:30:05.279969 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:30:05.279969 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:30:05.279969 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:30:05.279969 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:30:05.279969 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:30:05.279969 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:30:05.279969 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:30:05.279969 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:30:05.279969 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:30:05.279969 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:30:05.279969 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:30:05.279969 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 22:30:05.279969 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 22:30:05.279969 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 22:30:05.281077 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 22:30:05.281077 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:30:05.281077 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:30:05.281077 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:30:05.281077 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:30:05.281077 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:30:05.281077 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:30:05.281077 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:30:05.281077 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application with Lock( Jul 20 22:30:05.281077 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:30:05.281077 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:30:05.281077 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:30:05.281077 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:30:05.281077 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:30:05.281077 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:30:05.281077 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:30:05.281077 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:30:05.282917 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 22:30:05.282917 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 22:30:05.282917 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 22:30:05.282917 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 22:30:05.282917 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 22:30:05.282917 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 22:30:05.282917 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 22:30:05.282917 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: ERROR keystone.server.flask.application Jul 20 22:30:05.282917 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 645/1285] 10.222.0.81 () {56 vars in 1111 bytes} [Wed Jul 20 22:30:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:30:05.287584 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169dc0ab-e94a-4308-b500-92aad7a291a9 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:30:05.337833 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:30:05.388854 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169dc0ab-e94a-4308-b500-92aad7a291a9 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:30:05.421856 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e433d62-9bf2-406a-8ff4-377c04bac76c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:30:05.423174 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e433d62-9bf2-406a-8ff4-377c04bac76c None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:30:05.423461 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e433d62-9bf2-406a-8ff4-377c04bac76c None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:30:05.423745 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e433d62-9bf2-406a-8ff4-377c04bac76c None admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:30:05.424515 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e433d62-9bf2-406a-8ff4-377c04bac76c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:30:05.424826 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e433d62-9bf2-406a-8ff4-377c04bac76c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:30:05.426278 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e433d62-9bf2-406a-8ff4-377c04bac76c None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:30:05.438107 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 641/1286] 10.222.0.81 () {56 vars in 1148 bytes} [Wed Jul 20 22:30:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:30:05.444282 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581c24d3-0ae6-42ae-aac3-a7300ed26dba None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:30:05.542610 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581c24d3-0ae6-42ae-aac3-a7300ed26dba None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:30:05.575816 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82376193-9eb1-4e4c-9370-baabd1556636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:30:05.576460 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82376193-9eb1-4e4c-9370-baabd1556636 None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:30:05.576746 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82376193-9eb1-4e4c-9370-baabd1556636 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:30:05.577027 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82376193-9eb1-4e4c-9370-baabd1556636 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:30:05.577851 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82376193-9eb1-4e4c-9370-baabd1556636 None admin] RBAC: Adding query filter params (role.id=d89b6c92d35a401a8517f1d47f6be843, scope.project.id=385edf2e59e546d090ace36600e29032, user.id=d5500cebdd9d4f8e9aecb45d6c955878) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:30:05.578182 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82376193-9eb1-4e4c-9370-baabd1556636 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:30:05.580144 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82376193-9eb1-4e4c-9370-baabd1556636 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:30:05.585873 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 646/1287] 10.222.0.81 () {56 vars in 1374 bytes} [Wed Jul 20 22:30:05 2022] GET /identity/v3/role_assignments?user.id=d5500cebdd9d4f8e9aecb45d6c955878&scope.project.id=385edf2e59e546d090ace36600e29032&role.id=d89b6c92d35a401a8517f1d47f6be843 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:30:07.652730 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7780f8-d730-40d3-b4a1-94df6a608788 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:30:07.653568 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7780f8-d730-40d3-b4a1-94df6a608788 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:30:07.653568 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7780f8-d730-40d3-b4a1-94df6a608788 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:30:07.654912 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 642/1288] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:30:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:30:07.660681 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f50e94f-f931-4edc-8685-659b48004e0e None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:30:07.661090 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f50e94f-f931-4edc-8685-659b48004e0e None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:30:07.661236 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f50e94f-f931-4edc-8685-659b48004e0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:30:07.678292 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-9f50e94f-f931-4edc-8685-659b48004e0e None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:30:07.679500 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-9f50e94f-f931-4edc-8685-659b48004e0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:30:07.709287 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:30:07.744317 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 647/1289] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:30:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 22:30:07.767816 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7780f8-d730-40d3-b4a1-94df6a608788 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:30:07.866560 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7780f8-d730-40d3-b4a1-94df6a608788 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:30:07.877770 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:30:07.900187 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2384220c-1517-4750-bf4e-2ba7ec388975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:30:07.901000 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2384220c-1517-4750-bf4e-2ba7ec388975 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:30:07.901301 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2384220c-1517-4750-bf4e-2ba7ec388975 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:30:07.901581 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2384220c-1517-4750-bf4e-2ba7ec388975 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:30:07.902202 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2384220c-1517-4750-bf4e-2ba7ec388975 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:30:07.903909 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2384220c-1517-4750-bf4e-2ba7ec388975 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:30:07.915982 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-2384220c-1517-4750-bf4e-2ba7ec388975 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 20 22:30:07.915982 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:30:07.915982 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 22:30:07.915982 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 22:30:07.915982 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 22:30:07.915982 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 22:30:07.915982 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 22:30:07.915982 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 22:30:07.915982 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 20 22:30:07.915982 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:30:07.915982 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 22:30:07.915982 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:30:07.915982 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:30:07.915982 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:30:07.915982 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:30:07.915982 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:30:07.915982 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:30:07.917276 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:30:07.917276 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:30:07.917276 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:30:07.917276 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:30:07.917276 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:30:07.917276 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:30:07.917276 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 22:30:07.917276 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 22:30:07.917276 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 22:30:07.917276 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 22:30:07.917276 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:30:07.917276 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:30:07.917276 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:30:07.917276 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:30:07.917276 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:30:07.917276 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:30:07.918443 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:30:07.918443 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:30:07.918443 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:30:07.918443 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:30:07.918443 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:30:07.918443 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:30:07.918443 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:30:07.918443 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:30:07.918443 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:30:07.918443 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:30:07.918443 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 22:30:07.918443 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 22:30:07.918443 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 20 22:30:07.918443 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:30:07.918443 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 643/1290] 10.222.0.81 () {56 vars in 1117 bytes} [Wed Jul 20 22:30:07 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 22:30:07.923721 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f50e94f-f931-4edc-8685-659b48004e0e None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:30:08.021727 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f50e94f-f931-4edc-8685-659b48004e0e None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:30:08.056552 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd94dc2-a979-4da4-a2fa-9dbff28eef8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:30:08.057214 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd94dc2-a979-4da4-a2fa-9dbff28eef8e None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:30:08.057507 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd94dc2-a979-4da4-a2fa-9dbff28eef8e None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:30:08.057783 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd94dc2-a979-4da4-a2fa-9dbff28eef8e None admin] PATH_INFO: `/v3/services` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:30:08.058493 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd94dc2-a979-4da4-a2fa-9dbff28eef8e None admin] RBAC: Adding query filter params (name=placement) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:30:08.058802 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd94dc2-a979-4da4-a2fa-9dbff28eef8e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:30:08.060203 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd94dc2-a979-4da4-a2fa-9dbff28eef8e None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:30:08.065072 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 648/1291] 10.222.0.81 () {56 vars in 1116 bytes} [Wed Jul 20 22:30:07 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:30:08.070920 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2384220c-1517-4750-bf4e-2ba7ec388975 None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:30:08.170405 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2384220c-1517-4750-bf4e-2ba7ec388975 None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:30:08.202892 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f34bec1-3cba-4040-b819-7d083852ec82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:30:08.203595 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f34bec1-3cba-4040-b819-7d083852ec82 None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:30:08.203882 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f34bec1-3cba-4040-b819-7d083852ec82 None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:30:08.204696 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f34bec1-3cba-4040-b819-7d083852ec82 None admin] PATH_INFO: `/v3/services` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:30:08.205398 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f34bec1-3cba-4040-b819-7d083852ec82 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:30:08.205706 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f34bec1-3cba-4040-b819-7d083852ec82 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:30:08.207116 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f34bec1-3cba-4040-b819-7d083852ec82 None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:30:08.211710 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 644/1292] 10.222.0.81 () {56 vars in 1116 bytes} [Wed Jul 20 22:30:08 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:30:10.227971 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b8ba42-47a1-4352-b775-36cca88d26dc None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:30:10.228410 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b8ba42-47a1-4352-b775-36cca88d26dc None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:30:10.228542 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b8ba42-47a1-4352-b775-36cca88d26dc None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:30:10.229914 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 649/1293] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:30:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:30:10.235123 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21eda8c-f9d8-45e3-a434-eccb29d6a317 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:30:10.235539 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21eda8c-f9d8-45e3-a434-eccb29d6a317 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:30:10.236187 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21eda8c-f9d8-45e3-a434-eccb29d6a317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:30:10.253346 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-c21eda8c-f9d8-45e3-a434-eccb29d6a317 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:30:10.254582 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-c21eda8c-f9d8-45e3-a434-eccb29d6a317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:30:10.284352 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:30:10.321241 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 645/1294] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:30:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 22:30:10.353900 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b8ba42-47a1-4352-b775-36cca88d26dc None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:30:10.391425 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:30:10.461120 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b8ba42-47a1-4352-b775-36cca88d26dc None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:30:10.495361 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640c4e8b-3c3e-4e33-821b-a917524eb4a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:30:10.496024 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640c4e8b-3c3e-4e33-821b-a917524eb4a5 None admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:30:10.496311 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640c4e8b-3c3e-4e33-821b-a917524eb4a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:30:10.496595 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640c4e8b-3c3e-4e33-821b-a917524eb4a5 None admin] PATH_INFO: `/v3/services` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:30:10.497330 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640c4e8b-3c3e-4e33-821b-a917524eb4a5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:30:10.498695 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640c4e8b-3c3e-4e33-821b-a917524eb4a5 None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:30:10.606102 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 650/1295] 10.222.0.81 () {60 vars in 1141 bytes} [Wed Jul 20 22:30:10 2022] POST /identity/v3/services => generated 246 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:30:12.703389 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943ace1a-e5e0-4e69-90cb-fb2ca57e9b58 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:30:12.703922 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943ace1a-e5e0-4e69-90cb-fb2ca57e9b58 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:30:12.703922 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943ace1a-e5e0-4e69-90cb-fb2ca57e9b58 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:30:12.705404 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 646/1296] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:30:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:30:12.710821 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7c0f4e-a614-41b8-a847-58a1ac036929 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:30:12.711280 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7c0f4e-a614-41b8-a847-58a1ac036929 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:30:12.711433 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7c0f4e-a614-41b8-a847-58a1ac036929 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:30:12.727847 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-ab7c0f4e-a614-41b8-a847-58a1ac036929 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:30:12.729029 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-ab7c0f4e-a614-41b8-a847-58a1ac036929 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:30:12.801785 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 651/1297] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:30:12 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 22:30:12.825645 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943ace1a-e5e0-4e69-90cb-fb2ca57e9b58 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:30:12.924750 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943ace1a-e5e0-4e69-90cb-fb2ca57e9b58 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:30:12.958143 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cfd1ed-bb5a-4b36-948b-6164654ed90d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:30:12.958941 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cfd1ed-bb5a-4b36-948b-6164654ed90d None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:30:12.959292 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cfd1ed-bb5a-4b36-948b-6164654ed90d None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:30:12.959611 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cfd1ed-bb5a-4b36-948b-6164654ed90d None admin] PATH_INFO: `/v3/services/placement` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:30:12.960298 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cfd1ed-bb5a-4b36-948b-6164654ed90d None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:30:12.961769 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cfd1ed-bb5a-4b36-948b-6164654ed90d None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:30:12.975115 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-f0cfd1ed-bb5a-4b36-948b-6164654ed90d None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 20 22:30:12.975115 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:30:12.975115 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 22:30:12.975115 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 22:30:12.975115 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 22:30:12.975115 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 22:30:12.975115 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 22:30:12.975115 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 22:30:12.975115 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 20 22:30:12.975115 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:30:12.975115 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 22:30:12.975115 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:30:12.975115 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:30:12.975115 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:30:12.975115 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:30:12.975115 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:30:12.975115 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:30:12.976204 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:30:12.976204 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:30:12.976204 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:30:12.976204 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:30:12.976204 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:30:12.976204 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:30:12.976204 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 22:30:12.976204 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 22:30:12.976204 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 22:30:12.976204 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 22:30:12.976204 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:30:12.976204 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:30:12.976204 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:30:12.976204 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:30:12.976204 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:30:12.976204 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:30:12.977035 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:30:12.977035 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:30:12.977035 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:30:12.977035 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:30:12.977035 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:30:12.977035 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:30:12.977035 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:30:12.977035 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:30:12.977035 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:30:12.977035 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:30:12.977035 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 22:30:12.977035 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 22:30:12.977035 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 20 22:30:12.977035 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:30:12.977035 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 647/1298] 10.222.0.81 () {56 vars in 1117 bytes} [Wed Jul 20 22:30:12 2022] GET /identity/v3/services/placement => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:30:12.982567 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7c0f4e-a614-41b8-a847-58a1ac036929 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:30:13.000948 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:30:13.080068 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7c0f4e-a614-41b8-a847-58a1ac036929 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:30:13.113746 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee3a56c-5d33-4269-850c-b6ab2631d49b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:30:13.114412 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee3a56c-5d33-4269-850c-b6ab2631d49b None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:30:13.114702 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee3a56c-5d33-4269-850c-b6ab2631d49b None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:30:13.114986 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee3a56c-5d33-4269-850c-b6ab2631d49b None admin] PATH_INFO: `/v3/services` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:30:13.115724 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee3a56c-5d33-4269-850c-b6ab2631d49b None admin] RBAC: Adding query filter params (name=placement) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:30:13.116036 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee3a56c-5d33-4269-850c-b6ab2631d49b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:30:13.117437 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee3a56c-5d33-4269-850c-b6ab2631d49b None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:30:13.122421 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 652/1299] 10.222.0.81 () {56 vars in 1116 bytes} [Wed Jul 20 22:30:12 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:30:13.129125 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cfd1ed-bb5a-4b36-948b-6164654ed90d None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:30:13.161601 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:30:13.230058 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cfd1ed-bb5a-4b36-948b-6164654ed90d None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:30:13.263232 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5df485e-4d0e-446d-a5f4-a4e90ebc9aed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:30:13.263898 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5df485e-4d0e-446d-a5f4-a4e90ebc9aed None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:30:13.264188 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5df485e-4d0e-446d-a5f4-a4e90ebc9aed None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:30:13.264468 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5df485e-4d0e-446d-a5f4-a4e90ebc9aed None admin] PATH_INFO: `/v3/endpoints` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:30:13.265258 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5df485e-4d0e-446d-a5f4-a4e90ebc9aed None admin] RBAC: Adding query filter params (interface=public, service_id=20bb448653274e40bb47fccf1a026b45, region_id=RegionOne) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:30:13.265574 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5df485e-4d0e-446d-a5f4-a4e90ebc9aed None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:30:13.266961 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5df485e-4d0e-446d-a5f4-a4e90ebc9aed None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:30:13.272159 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 648/1300] 10.222.0.81 () {56 vars in 1251 bytes} [Wed Jul 20 22:30:13 2022] GET /identity/v3/endpoints?service_id=20bb448653274e40bb47fccf1a026b45&interface=public®ion_id=RegionOne => generated 194 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:30:13.277809 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee3a56c-5d33-4269-850c-b6ab2631d49b None admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:30:13.359536 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:30:13.377740 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee3a56c-5d33-4269-850c-b6ab2631d49b None admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:30:13.416290 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665a90b3-e2ee-4b10-aab2-67fd3be53c8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:30:13.416958 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665a90b3-e2ee-4b10-aab2-67fd3be53c8a None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:30:13.417249 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665a90b3-e2ee-4b10-aab2-67fd3be53c8a None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:30:13.417636 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665a90b3-e2ee-4b10-aab2-67fd3be53c8a None admin] PATH_INFO: `/v3/services` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:30:13.418324 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665a90b3-e2ee-4b10-aab2-67fd3be53c8a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:30:13.419726 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665a90b3-e2ee-4b10-aab2-67fd3be53c8a None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:30:13.426671 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 653/1301] 10.222.0.81 () {56 vars in 1087 bytes} [Wed Jul 20 22:30:13 2022] GET /identity/v3/services => generated 2191 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:30:15.528269 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e14ca1-abd8-440b-84b0-924b407194a4 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:30:15.528769 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e14ca1-abd8-440b-84b0-924b407194a4 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:30:15.528769 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e14ca1-abd8-440b-84b0-924b407194a4 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:30:15.530569 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 649/1302] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:30:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:30:15.536309 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95af6294-3122-4686-b10b-cd5db3b67d40 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:30:15.536765 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95af6294-3122-4686-b10b-cd5db3b67d40 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:30:15.536855 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95af6294-3122-4686-b10b-cd5db3b67d40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:30:15.554001 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-95af6294-3122-4686-b10b-cd5db3b67d40 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:30:15.555445 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-95af6294-3122-4686-b10b-cd5db3b67d40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:30:15.630849 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 654/1303] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:30:15 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 22:30:15.653832 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e14ca1-abd8-440b-84b0-924b407194a4 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:30:15.753874 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:30:15.760757 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e14ca1-abd8-440b-84b0-924b407194a4 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:30:15.804163 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14834daa-6f2a-448e-a5fd-46a27ebc87ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:30:15.804926 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14834daa-6f2a-448e-a5fd-46a27ebc87ec None admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:30:15.805212 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14834daa-6f2a-448e-a5fd-46a27ebc87ec None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:30:15.805488 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14834daa-6f2a-448e-a5fd-46a27ebc87ec None admin] PATH_INFO: `/v3/services/placement` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:30:15.806131 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14834daa-6f2a-448e-a5fd-46a27ebc87ec None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:30:15.807532 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14834daa-6f2a-448e-a5fd-46a27ebc87ec None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:30:15.820403 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application [None req-14834daa-6f2a-448e-a5fd-46a27ebc87ec None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 20 22:30:15.820403 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:30:15.820403 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 22:30:15.820403 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 22:30:15.820403 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 22:30:15.820403 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 22:30:15.820403 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 22:30:15.820403 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 22:30:15.820403 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 20 22:30:15.820403 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:30:15.820403 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 22:30:15.820403 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:30:15.820403 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 22:30:15.820403 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 22:30:15.820403 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 22:30:15.820403 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 22:30:15.820403 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 22:30:15.821602 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 22:30:15.821602 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 22:30:15.821602 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 22:30:15.821602 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 22:30:15.821602 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 22:30:15.821602 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 22:30:15.821602 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 22:30:15.821602 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 22:30:15.821602 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 22:30:15.821602 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 22:30:15.821602 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 22:30:15.821602 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 22:30:15.821602 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 22:30:15.821602 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 22:30:15.821602 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 22:30:15.821602 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 22:30:15.822776 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 22:30:15.822776 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application with Lock( Jul 20 22:30:15.822776 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 22:30:15.822776 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self._enter() Jul 20 22:30:15.822776 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 22:30:15.822776 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 22:30:15.822776 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 22:30:15.822776 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application return self.creator() Jul 20 22:30:15.822776 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 22:30:15.822776 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application created_value = creator( Jul 20 22:30:15.822776 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 22:30:15.822776 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 22:30:15.822776 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 20 22:30:15.822776 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: ERROR keystone.server.flask.application Jul 20 22:30:15.822776 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 650/1304] 10.222.0.81 () {56 vars in 1117 bytes} [Wed Jul 20 22:30:15 2022] GET /identity/v3/services/placement => generated 90 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 22:30:15.827720 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95af6294-3122-4686-b10b-cd5db3b67d40 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:30:15.905639 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:30:15.927810 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95af6294-3122-4686-b10b-cd5db3b67d40 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:30:15.961443 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c05b52-fbe5-4311-9746-98bec534524c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:30:15.962112 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c05b52-fbe5-4311-9746-98bec534524c None admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:30:15.962397 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c05b52-fbe5-4311-9746-98bec534524c None admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:30:15.962709 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c05b52-fbe5-4311-9746-98bec534524c None admin] PATH_INFO: `/v3/services` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:30:15.963486 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c05b52-fbe5-4311-9746-98bec534524c None admin] RBAC: Adding query filter params (name=placement) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:30:15.963805 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c05b52-fbe5-4311-9746-98bec534524c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:30:15.965212 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c05b52-fbe5-4311-9746-98bec534524c None admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:30:15.970435 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 655/1305] 10.222.0.81 () {56 vars in 1116 bytes} [Wed Jul 20 22:30:15 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 22:30:15.976273 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14834daa-6f2a-448e-a5fd-46a27ebc87ec None admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:30:16.082023 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14834daa-6f2a-448e-a5fd-46a27ebc87ec None admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:30:16.123564 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf8590f-6f06-4016-b8d8-2a26a40578fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:30:16.124342 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf8590f-6f06-4016-b8d8-2a26a40578fa None admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:30:16.124629 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf8590f-6f06-4016-b8d8-2a26a40578fa None admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:30:16.124910 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf8590f-6f06-4016-b8d8-2a26a40578fa None admin] PATH_INFO: `/v3/endpoints` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:30:16.125667 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf8590f-6f06-4016-b8d8-2a26a40578fa None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:30:16.127073 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf8590f-6f06-4016-b8d8-2a26a40578fa None admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:30:16.200567 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 651/1306] 10.222.0.81 () {60 vars in 1144 bytes} [Wed Jul 20 22:30:15 2022] POST /identity/v3/endpoints => generated 331 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:30:25.988433 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=30 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:30:26.195458 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:36:34.643138 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b3c9d0-4184-4372-9242-14c1f6a1851f None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:36:34.643138 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b3c9d0-4184-4372-9242-14c1f6a1851f None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:36:34.644086 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b3c9d0-4184-4372-9242-14c1f6a1851f None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:36:34.651437 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391eeffb-cee7-4038-980d-a9c33eb5d9d8 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:36:34.651948 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391eeffb-cee7-4038-980d-a9c33eb5d9d8 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:36:34.652033 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391eeffb-cee7-4038-980d-a9c33eb5d9d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:36:34.653875 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 656/1307] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:36:34 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:36:34.742237 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-391eeffb-cee7-4038-980d-a9c33eb5d9d8 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:36:34.744240 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-391eeffb-cee7-4038-980d-a9c33eb5d9d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:36:34.883183 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 652/1308] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:36:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:36:34.908466 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebaf088-61ae-4686-8d1f-3ebb654c7924 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:36:34.908964 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebaf088-61ae-4686-8d1f-3ebb654c7924 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:36:34.909440 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebaf088-61ae-4686-8d1f-3ebb654c7924 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:36:34.911624 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 657/1309] 10.222.0.81 () {54 vars in 965 bytes} [Wed Jul 20 22:36:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:36:34.918497 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848c70d6-862b-4533-b71d-54155795ee51 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:36:34.919736 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848c70d6-862b-4533-b71d-54155795ee51 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:36:34.920029 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848c70d6-862b-4533-b71d-54155795ee51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:36:35.080144 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-848c70d6-862b-4533-b71d-54155795ee51 None None] MFA Rules not processed for user `333c2b1b11cc4c478abfd965a72e9d66`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:36:35.081744 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-848c70d6-862b-4533-b71d-54155795ee51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:36:35.109442 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:36:35.170361 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 653/1310] 10.222.0.81 () {58 vars in 1062 bytes} [Wed Jul 20 22:36:34 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:36:35.179225 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebaf088-61ae-4686-8d1f-3ebb654c7924 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:36:35.182226 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-4ebaf088-61ae-4686-8d1f-3ebb654c7924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:36:35.302256 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebaf088-61ae-4686-8d1f-3ebb654c7924 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:36:35.382485 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42169ed-977e-4856-80c7-dbaa3d8e8b24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:36:35.383236 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42169ed-977e-4856-80c7-dbaa3d8e8b24 service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:36:35.383473 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42169ed-977e-4856-80c7-dbaa3d8e8b24 service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:36:35.383779 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42169ed-977e-4856-80c7-dbaa3d8e8b24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:36:35.385413 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-b42169ed-977e-4856-80c7-dbaa3d8e8b24 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:36:35.442222 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:36:35.498793 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42169ed-977e-4856-80c7-dbaa3d8e8b24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:36:35.500276 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42169ed-977e-4856-80c7-dbaa3d8e8b24 service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:36:35.520301 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 658/1311] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:36:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:36:37.866176 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73645510-b53c-4030-8aa7-3ab1a00ea26e None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:36:37.866714 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73645510-b53c-4030-8aa7-3ab1a00ea26e None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:36:37.866820 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73645510-b53c-4030-8aa7-3ab1a00ea26e None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:36:37.868827 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 654/1312] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:36:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:36:37.873884 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259c835a-4bcc-408a-83a7-b7718f3da0c4 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:36:37.874285 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259c835a-4bcc-408a-83a7-b7718f3da0c4 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:36:37.874508 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259c835a-4bcc-408a-83a7-b7718f3da0c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:36:37.955701 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-259c835a-4bcc-408a-83a7-b7718f3da0c4 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:36:37.956932 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-259c835a-4bcc-408a-83a7-b7718f3da0c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:36:38.068676 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 659/1313] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:36:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:36:38.108787 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73645510-b53c-4030-8aa7-3ab1a00ea26e None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:36:38.110451 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-73645510-b53c-4030-8aa7-3ab1a00ea26e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:36:38.234366 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73645510-b53c-4030-8aa7-3ab1a00ea26e None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:36:38.319787 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:36:38.323108 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dd78b8-84e8-452f-8230-276375d94e57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:36:38.324034 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dd78b8-84e8-452f-8230-276375d94e57 service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:36:38.324546 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dd78b8-84e8-452f-8230-276375d94e57 service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:36:38.324936 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dd78b8-84e8-452f-8230-276375d94e57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:36:38.326827 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-90dd78b8-84e8-452f-8230-276375d94e57 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:36:38.441897 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dd78b8-84e8-452f-8230-276375d94e57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:36:38.443375 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dd78b8-84e8-452f-8230-276375d94e57 service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:36:38.461499 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 655/1314] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:36:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 22:36:41.019348 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3459333-6518-4a19-9b07-cf72215e187d None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:36:41.019949 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3459333-6518-4a19-9b07-cf72215e187d None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:36:41.019949 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3459333-6518-4a19-9b07-cf72215e187d None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:36:41.021274 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 660/1315] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:36:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:36:41.027119 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d58cb0e-24e7-4906-8cac-61c1aedc965b None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:36:41.027524 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d58cb0e-24e7-4906-8cac-61c1aedc965b None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:36:41.027687 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d58cb0e-24e7-4906-8cac-61c1aedc965b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:36:41.090864 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-7d58cb0e-24e7-4906-8cac-61c1aedc965b None None] MFA Rules not processed for user `74bf4b54c7664f9fac20de5b75d465f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:36:41.092120 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-7d58cb0e-24e7-4906-8cac-61c1aedc965b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:36:41.143610 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:36:41.189317 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 656/1316] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:36:41 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:36:41.229024 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3459333-6518-4a19-9b07-cf72215e187d None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:36:41.230582 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-f3459333-6518-4a19-9b07-cf72215e187d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:36:41.240197 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:36:41.333180 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3459333-6518-4a19-9b07-cf72215e187d None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:36:41.411120 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ee8ad0-545e-4dfb-a243-cf29b1a40982 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:36:41.411805 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ee8ad0-545e-4dfb-a243-cf29b1a40982 service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:36:41.412103 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ee8ad0-545e-4dfb-a243-cf29b1a40982 service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:36:41.412384 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ee8ad0-545e-4dfb-a243-cf29b1a40982 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:36:41.414032 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-08ee8ad0-545e-4dfb-a243-cf29b1a40982 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:36:41.477384 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:36:41.522360 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ee8ad0-545e-4dfb-a243-cf29b1a40982 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:36:41.523821 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ee8ad0-545e-4dfb-a243-cf29b1a40982 service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:36:41.541901 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 661/1317] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:36:41 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:36:44.313976 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eebc4bd-bfc8-4be5-9483-d0c0165ed990 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:36:44.314505 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eebc4bd-bfc8-4be5-9483-d0c0165ed990 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:36:44.314614 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eebc4bd-bfc8-4be5-9483-d0c0165ed990 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:36:44.315963 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 657/1318] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:36:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:36:44.321574 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2725e55-a8ca-453a-876b-ccfa687f1c5d None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:36:44.321977 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2725e55-a8ca-453a-876b-ccfa687f1c5d None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:36:44.322141 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2725e55-a8ca-453a-876b-ccfa687f1c5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:36:44.386622 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-f2725e55-a8ca-453a-876b-ccfa687f1c5d None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:36:44.387870 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-f2725e55-a8ca-453a-876b-ccfa687f1c5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:36:44.482171 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 662/1319] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:36:44 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:36:44.521567 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eebc4bd-bfc8-4be5-9483-d0c0165ed990 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:36:44.523113 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-6eebc4bd-bfc8-4be5-9483-d0c0165ed990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:36:44.623385 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eebc4bd-bfc8-4be5-9483-d0c0165ed990 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:36:44.699794 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ffe124-c341-42c2-bbff-ba00d4b0d0a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:36:44.700455 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ffe124-c341-42c2-bbff-ba00d4b0d0a8 service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:36:44.700740 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ffe124-c341-42c2-bbff-ba00d4b0d0a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:36:44.701019 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ffe124-c341-42c2-bbff-ba00d4b0d0a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:36:44.702650 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-74ffe124-c341-42c2-bbff-ba00d4b0d0a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:36:44.709585 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:36:44.808516 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ffe124-c341-42c2-bbff-ba00d4b0d0a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:36:44.809942 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ffe124-c341-42c2-bbff-ba00d4b0d0a8 service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:36:44.827949 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 658/1320] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:36:44 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:36:47.977681 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b735391-0295-4976-b41f-580fa0cc6251 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:36:47.978226 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b735391-0295-4976-b41f-580fa0cc6251 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:36:47.978226 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b735391-0295-4976-b41f-580fa0cc6251 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:36:47.979644 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 663/1321] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:36:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:36:47.984797 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e7570f-a3f2-406c-98c1-3045d555d09d None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:36:47.985391 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e7570f-a3f2-406c-98c1-3045d555d09d None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:36:47.985391 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e7570f-a3f2-406c-98c1-3045d555d09d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:36:48.049836 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-24e7570f-a3f2-406c-98c1-3045d555d09d None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:36:48.051070 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-24e7570f-a3f2-406c-98c1-3045d555d09d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:36:48.109051 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:36:48.148564 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 659/1322] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:36:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:36:48.186743 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b735391-0295-4976-b41f-580fa0cc6251 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:36:48.188474 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-5b735391-0295-4976-b41f-580fa0cc6251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:36:48.212115 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:36:48.303746 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b735391-0295-4976-b41f-580fa0cc6251 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:36:48.384136 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71ce526-fe4b-4d03-8f5b-7b1fea481e80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:36:48.384800 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71ce526-fe4b-4d03-8f5b-7b1fea481e80 service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:36:48.385087 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71ce526-fe4b-4d03-8f5b-7b1fea481e80 service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:36:48.385370 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71ce526-fe4b-4d03-8f5b-7b1fea481e80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:36:48.387031 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-a71ce526-fe4b-4d03-8f5b-7b1fea481e80 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:36:48.463817 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:36:48.501699 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71ce526-fe4b-4d03-8f5b-7b1fea481e80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:36:48.503173 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71ce526-fe4b-4d03-8f5b-7b1fea481e80 service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:36:48.521904 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 664/1323] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:36:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:36:50.639443 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ad0357-c1d9-4970-9594-1b7073003343 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:36:50.640012 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ad0357-c1d9-4970-9594-1b7073003343 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:36:50.640012 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ad0357-c1d9-4970-9594-1b7073003343 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:36:50.641389 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 660/1324] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:36:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:36:50.647355 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e313f23-b64f-4eca-a0ea-61f18027e9df None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:36:50.647757 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e313f23-b64f-4eca-a0ea-61f18027e9df None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:36:50.647927 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e313f23-b64f-4eca-a0ea-61f18027e9df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:36:50.712294 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-5e313f23-b64f-4eca-a0ea-61f18027e9df None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:36:50.713421 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-5e313f23-b64f-4eca-a0ea-61f18027e9df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:36:50.809436 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 665/1325] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:36:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:36:50.845109 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ad0357-c1d9-4970-9594-1b7073003343 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:36:50.846625 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-09ad0357-c1d9-4970-9594-1b7073003343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:36:50.949323 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ad0357-c1d9-4970-9594-1b7073003343 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:36:51.023542 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051769d8-8cfc-4479-945e-689d6b81b19c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:36:51.024226 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051769d8-8cfc-4479-945e-689d6b81b19c service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:36:51.024511 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051769d8-8cfc-4479-945e-689d6b81b19c service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:36:51.024785 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051769d8-8cfc-4479-945e-689d6b81b19c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:36:51.026460 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-051769d8-8cfc-4479-945e-689d6b81b19c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:36:51.043796 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:36:51.141192 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051769d8-8cfc-4479-945e-689d6b81b19c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:36:51.142661 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051769d8-8cfc-4479-945e-689d6b81b19c service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:36:51.160183 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 661/1326] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:36:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:36:53.256940 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c2dbd3-c70c-4a4f-8703-e1f15f894ee1 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:36:53.257504 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c2dbd3-c70c-4a4f-8703-e1f15f894ee1 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:36:53.258141 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c2dbd3-c70c-4a4f-8703-e1f15f894ee1 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:36:53.259635 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 666/1327] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:36:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:36:53.265007 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d646cf08-7522-453b-af8a-d203592b1d7d None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:36:53.265621 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d646cf08-7522-453b-af8a-d203592b1d7d None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:36:53.265621 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d646cf08-7522-453b-af8a-d203592b1d7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:36:53.329461 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-d646cf08-7522-453b-af8a-d203592b1d7d None None] MFA Rules not processed for user `74bf4b54c7664f9fac20de5b75d465f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:36:53.330672 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-d646cf08-7522-453b-af8a-d203592b1d7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:36:53.396001 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:36:53.429349 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 662/1328] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:36:53 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 22:36:53.468723 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c2dbd3-c70c-4a4f-8703-e1f15f894ee1 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:36:53.470268 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-10c2dbd3-c70c-4a4f-8703-e1f15f894ee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:36:53.504303 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:36:53.582445 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c2dbd3-c70c-4a4f-8703-e1f15f894ee1 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:36:53.654652 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f9c44b-5fa5-4be3-a4d7-15f7cad765ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:36:53.655256 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f9c44b-5fa5-4be3-a4d7-15f7cad765ab service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:36:53.655544 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f9c44b-5fa5-4be3-a4d7-15f7cad765ab service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:36:53.655829 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f9c44b-5fa5-4be3-a4d7-15f7cad765ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:36:53.657486 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-d6f9c44b-5fa5-4be3-a4d7-15f7cad765ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:36:53.736522 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:36:53.767700 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f9c44b-5fa5-4be3-a4d7-15f7cad765ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:36:53.769146 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f9c44b-5fa5-4be3-a4d7-15f7cad765ab service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:36:53.787893 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 667/1329] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:36:53 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:36:56.309476 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7485d1c0-b61f-4719-bff1-983bd05fe41f None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:36:56.310025 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7485d1c0-b61f-4719-bff1-983bd05fe41f None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:36:56.310025 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7485d1c0-b61f-4719-bff1-983bd05fe41f None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:36:56.311441 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 663/1330] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:36:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:36:56.316571 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b2b052-3c8a-4fba-88fc-3b6bc3de2551 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:36:56.316955 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b2b052-3c8a-4fba-88fc-3b6bc3de2551 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:36:56.317176 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b2b052-3c8a-4fba-88fc-3b6bc3de2551 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:36:56.383111 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-a2b2b052-3c8a-4fba-88fc-3b6bc3de2551 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:36:56.384336 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-a2b2b052-3c8a-4fba-88fc-3b6bc3de2551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:36:56.482456 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 668/1331] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:36:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:36:56.518990 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7485d1c0-b61f-4719-bff1-983bd05fe41f None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:36:56.520632 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-7485d1c0-b61f-4719-bff1-983bd05fe41f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:36:56.621373 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7485d1c0-b61f-4719-bff1-983bd05fe41f None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:36:56.691754 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ef4a4d-2945-4699-bb33-e199bd7fc436 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:36:56.692408 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ef4a4d-2945-4699-bb33-e199bd7fc436 service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:36:56.692691 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ef4a4d-2945-4699-bb33-e199bd7fc436 service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:36:56.692968 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ef4a4d-2945-4699-bb33-e199bd7fc436 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:36:56.694590 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-75ef4a4d-2945-4699-bb33-e199bd7fc436 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:36:56.721964 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:36:56.802008 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ef4a4d-2945-4699-bb33-e199bd7fc436 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:36:56.803462 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ef4a4d-2945-4699-bb33-e199bd7fc436 service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:36:56.822571 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 664/1332] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:36:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:36:58.897463 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7515635c-76ae-4483-9f73-66103d9c65a7 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:36:58.898505 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7515635c-76ae-4483-9f73-66103d9c65a7 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:36:58.898622 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7515635c-76ae-4483-9f73-66103d9c65a7 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:36:58.899983 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 669/1333] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:36:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:36:58.905095 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0247d8b1-e264-4492-a26b-a400c0427f40 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:36:58.905519 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0247d8b1-e264-4492-a26b-a400c0427f40 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:36:58.905673 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0247d8b1-e264-4492-a26b-a400c0427f40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:36:58.968261 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-0247d8b1-e264-4492-a26b-a400c0427f40 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:36:58.969483 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-0247d8b1-e264-4492-a26b-a400c0427f40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:36:59.042978 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:36:59.064333 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 665/1334] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:36:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:36:59.105527 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7515635c-76ae-4483-9f73-66103d9c65a7 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:36:59.107128 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-7515635c-76ae-4483-9f73-66103d9c65a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:36:59.147116 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:36:59.211587 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7515635c-76ae-4483-9f73-66103d9c65a7 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:36:59.283502 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2783f8f4-f3a4-403e-b3d3-dbfe46a0a35f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:36:59.284143 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2783f8f4-f3a4-403e-b3d3-dbfe46a0a35f service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:36:59.284423 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2783f8f4-f3a4-403e-b3d3-dbfe46a0a35f service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:36:59.284729 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2783f8f4-f3a4-403e-b3d3-dbfe46a0a35f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:36:59.286343 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-2783f8f4-f3a4-403e-b3d3-dbfe46a0a35f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:36:59.366327 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:36:59.391555 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2783f8f4-f3a4-403e-b3d3-dbfe46a0a35f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:36:59.393007 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2783f8f4-f3a4-403e-b3d3-dbfe46a0a35f service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:36:59.411444 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 670/1335] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:36:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:37:01.981311 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a43bbd4-3fee-4387-a71b-303bb3ba080d None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:37:01.981803 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a43bbd4-3fee-4387-a71b-303bb3ba080d None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:37:01.981923 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a43bbd4-3fee-4387-a71b-303bb3ba080d None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:37:01.983290 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 666/1336] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:37:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:37:01.988526 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e6f2ea-d99d-41db-9e40-b96ec8fe993d None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:37:01.988977 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e6f2ea-d99d-41db-9e40-b96ec8fe993d None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:37:01.989121 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e6f2ea-d99d-41db-9e40-b96ec8fe993d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:37:02.054633 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-32e6f2ea-d99d-41db-9e40-b96ec8fe993d None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:37:02.055864 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-32e6f2ea-d99d-41db-9e40-b96ec8fe993d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:37:02.149861 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 671/1337] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:37:01 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:37:02.187656 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a43bbd4-3fee-4387-a71b-303bb3ba080d None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:37:02.189200 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-8a43bbd4-3fee-4387-a71b-303bb3ba080d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:37:02.291163 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a43bbd4-3fee-4387-a71b-303bb3ba080d None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:37:02.365285 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f9f896-c23f-43e2-9e50-f9576cd2055e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:37:02.365943 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f9f896-c23f-43e2-9e50-f9576cd2055e service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:37:02.366227 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f9f896-c23f-43e2-9e50-f9576cd2055e service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:37:02.366499 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f9f896-c23f-43e2-9e50-f9576cd2055e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:37:02.368176 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-13f9f896-c23f-43e2-9e50-f9576cd2055e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:37:02.403480 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:37:02.474344 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f9f896-c23f-43e2-9e50-f9576cd2055e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:37:02.476150 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f9f896-c23f-43e2-9e50-f9576cd2055e service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:37:02.494725 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 667/1338] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:37:02 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:37:06.781130 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac10a3a-5b14-40ae-8cf5-e5b6d96f7191 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:37:06.781645 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac10a3a-5b14-40ae-8cf5-e5b6d96f7191 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:37:06.781645 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac10a3a-5b14-40ae-8cf5-e5b6d96f7191 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:37:06.783189 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 672/1339] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:37:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:37:06.789087 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985393a8-1066-48ac-a8af-1de711dc6fdf None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:37:06.789795 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985393a8-1066-48ac-a8af-1de711dc6fdf None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:37:06.789886 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985393a8-1066-48ac-a8af-1de711dc6fdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:37:06.862245 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-985393a8-1066-48ac-a8af-1de711dc6fdf None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:37:06.863613 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-985393a8-1066-48ac-a8af-1de711dc6fdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:37:06.958865 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:37:06.967236 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 668/1340] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:37:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:37:07.006825 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac10a3a-5b14-40ae-8cf5-e5b6d96f7191 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:37:07.009111 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-0ac10a3a-5b14-40ae-8cf5-e5b6d96f7191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:37:07.058915 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:37:07.114149 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac10a3a-5b14-40ae-8cf5-e5b6d96f7191 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:37:07.188522 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547e2762-e9ed-4999-8001-737b9f7aa174 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:37:07.189111 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547e2762-e9ed-4999-8001-737b9f7aa174 service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:37:07.189406 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547e2762-e9ed-4999-8001-737b9f7aa174 service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:37:07.189690 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547e2762-e9ed-4999-8001-737b9f7aa174 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:37:07.191382 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-547e2762-e9ed-4999-8001-737b9f7aa174 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:37:07.287574 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:37:07.305000 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547e2762-e9ed-4999-8001-737b9f7aa174 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:37:07.306921 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547e2762-e9ed-4999-8001-737b9f7aa174 service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:37:07.327624 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 673/1341] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:37:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:37:10.461284 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d571b1-215a-4c2b-a6de-de422218f381 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:37:10.461787 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d571b1-215a-4c2b-a6de-de422218f381 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:37:10.461787 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d571b1-215a-4c2b-a6de-de422218f381 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:37:10.463284 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 669/1342] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:37:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:37:10.468490 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66926d9-68f3-4d67-be8f-4f66fd645cfe None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:37:10.471978 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66926d9-68f3-4d67-be8f-4f66fd645cfe None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:37:10.471978 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66926d9-68f3-4d67-be8f-4f66fd645cfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:37:10.535386 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-c66926d9-68f3-4d67-be8f-4f66fd645cfe None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:37:10.536502 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-c66926d9-68f3-4d67-be8f-4f66fd645cfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:37:10.632937 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 674/1343] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:37:10 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:37:10.686671 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245f3a4f-6eae-4982-a2a4-6e7f8c8255f1 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:37:10.686959 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245f3a4f-6eae-4982-a2a4-6e7f8c8255f1 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:37:10.687264 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245f3a4f-6eae-4982-a2a4-6e7f8c8255f1 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:37:10.688656 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 670/1344] 10.222.0.81 () {54 vars in 965 bytes} [Wed Jul 20 22:37:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:37:10.694820 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed46de2c-a9bc-45de-8455-7a6231dadf11 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:37:10.695217 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed46de2c-a9bc-45de-8455-7a6231dadf11 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:37:10.695570 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed46de2c-a9bc-45de-8455-7a6231dadf11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:37:10.761837 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:37:10.769958 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-ed46de2c-a9bc-45de-8455-7a6231dadf11 None None] MFA Rules not processed for user `333c2b1b11cc4c478abfd965a72e9d66`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:37:10.775048 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-ed46de2c-a9bc-45de-8455-7a6231dadf11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:37:10.849539 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 675/1345] 10.222.0.81 () {58 vars in 1062 bytes} [Wed Jul 20 22:37:10 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:37:10.858964 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245f3a4f-6eae-4982-a2a4-6e7f8c8255f1 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:37:10.860513 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-245f3a4f-6eae-4982-a2a4-6e7f8c8255f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:37:10.967053 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245f3a4f-6eae-4982-a2a4-6e7f8c8255f1 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:37:11.039965 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d825d8-b126-451b-95ea-16587ca31649 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:37:11.040545 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d825d8-b126-451b-95ea-16587ca31649 service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:37:11.040832 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d825d8-b126-451b-95ea-16587ca31649 service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:37:11.041115 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d825d8-b126-451b-95ea-16587ca31649 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:37:11.042751 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-33d825d8-b126-451b-95ea-16587ca31649 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:37:11.092532 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:37:11.161262 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d825d8-b126-451b-95ea-16587ca31649 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:37:11.162704 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d825d8-b126-451b-95ea-16587ca31649 service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:37:11.181584 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 671/1346] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:37:10 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:37:16.297585 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c0b72f-a3ca-4a4e-a648-306de8682486 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:37:16.298162 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c0b72f-a3ca-4a4e-a648-306de8682486 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:37:16.298162 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c0b72f-a3ca-4a4e-a648-306de8682486 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:37:16.299575 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 676/1347] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:37:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:37:16.305661 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e14128f-e7aa-4fc6-99f8-389a23acfbe2 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:37:16.306169 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e14128f-e7aa-4fc6-99f8-389a23acfbe2 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:37:16.306272 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e14128f-e7aa-4fc6-99f8-389a23acfbe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:37:16.378534 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-3e14128f-e7aa-4fc6-99f8-389a23acfbe2 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:37:16.379788 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-3e14128f-e7aa-4fc6-99f8-389a23acfbe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:37:16.501305 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:37:16.503911 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 672/1348] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:37:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:37:16.543208 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c0b72f-a3ca-4a4e-a648-306de8682486 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:37:16.544783 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-58c0b72f-a3ca-4a4e-a648-306de8682486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:37:16.666653 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c0b72f-a3ca-4a4e-a648-306de8682486 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:37:16.716260 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:37:16.742064 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abc669f-12d7-42ca-87d5-339d03c328ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:37:16.742910 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abc669f-12d7-42ca-87d5-339d03c328ac service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:37:16.743477 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abc669f-12d7-42ca-87d5-339d03c328ac service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:37:16.743768 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abc669f-12d7-42ca-87d5-339d03c328ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:37:16.745424 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-2abc669f-12d7-42ca-87d5-339d03c328ac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:37:16.853498 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abc669f-12d7-42ca-87d5-339d03c328ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:37:16.854873 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abc669f-12d7-42ca-87d5-339d03c328ac service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:37:16.873080 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 677/1349] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:37:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:37:21.749912 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d40111-b47c-467f-8fd7-67864342c51c None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:37:21.750342 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d40111-b47c-467f-8fd7-67864342c51c None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:37:21.750464 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d40111-b47c-467f-8fd7-67864342c51c None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:37:21.752184 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 673/1350] 10.222.0.81 () {54 vars in 921 bytes} [Wed Jul 20 22:37:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:37:21.757705 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462aa65f-3231-43ab-9ce1-c06d32d1c8ac None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:37:21.758169 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462aa65f-3231-43ab-9ce1-c06d32d1c8ac None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:37:21.758267 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462aa65f-3231-43ab-9ce1-c06d32d1c8ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:37:21.880398 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-462aa65f-3231-43ab-9ce1-c06d32d1c8ac None None] MFA Rules not processed for user `d5500cebdd9d4f8e9aecb45d6c955878`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:37:21.882436 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-462aa65f-3231-43ab-9ce1-c06d32d1c8ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:37:21.907065 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:37:22.016961 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 678/1351] 10.222.0.81 () {58 vars in 1018 bytes} [Wed Jul 20 22:37:21 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:37:26.183982 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878adfb1-ae2b-4968-abfb-539beee50213 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:37:26.183982 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878adfb1-ae2b-4968-abfb-539beee50213 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:37:26.183982 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878adfb1-ae2b-4968-abfb-539beee50213 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:37:26.186258 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 674/1352] 10.222.0.81 () {54 vars in 921 bytes} [Wed Jul 20 22:37:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:37:26.193834 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719a69cc-f989-4cc9-ac77-13805ae7d1e0 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:37:26.195180 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719a69cc-f989-4cc9-ac77-13805ae7d1e0 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:37:26.195610 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719a69cc-f989-4cc9-ac77-13805ae7d1e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:37:26.274885 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-719a69cc-f989-4cc9-ac77-13805ae7d1e0 None None] MFA Rules not processed for user `d5500cebdd9d4f8e9aecb45d6c955878`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:37:26.276119 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-719a69cc-f989-4cc9-ac77-13805ae7d1e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:37:26.351596 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:37:26.394518 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 679/1353] 10.222.0.81 () {58 vars in 1018 bytes} [Wed Jul 20 22:37:26 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:37:27.697429 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d863a07c-19de-406b-9e78-f8a3686de4ec None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:37:27.697995 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d863a07c-19de-406b-9e78-f8a3686de4ec None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:37:27.698250 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d863a07c-19de-406b-9e78-f8a3686de4ec None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:37:27.699765 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 675/1354] 10.222.0.81 () {54 vars in 921 bytes} [Wed Jul 20 22:37:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:37:27.705186 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53400fe5-e49f-4605-b9b6-1c582af8cced None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:37:27.705652 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53400fe5-e49f-4605-b9b6-1c582af8cced None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:37:27.705878 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53400fe5-e49f-4605-b9b6-1c582af8cced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:37:27.787646 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-53400fe5-e49f-4605-b9b6-1c582af8cced None None] MFA Rules not processed for user `d5500cebdd9d4f8e9aecb45d6c955878`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:37:27.789123 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-53400fe5-e49f-4605-b9b6-1c582af8cced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:37:27.902121 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 680/1355] 10.222.0.81 () {58 vars in 1018 bytes} [Wed Jul 20 22:37:27 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:37:31.329020 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055025fc-a597-4aa9-89a6-22d4e4446c79 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:37:31.329552 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055025fc-a597-4aa9-89a6-22d4e4446c79 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:37:31.329552 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055025fc-a597-4aa9-89a6-22d4e4446c79 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:37:31.334863 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 676/1356] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:37:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:37:31.337433 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab184027-c06b-45e2-b5a2-c75e70326776 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:37:31.337879 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab184027-c06b-45e2-b5a2-c75e70326776 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:37:31.338026 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab184027-c06b-45e2-b5a2-c75e70326776 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:37:31.386242 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:37:31.433052 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-ab184027-c06b-45e2-b5a2-c75e70326776 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:37:31.434561 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-ab184027-c06b-45e2-b5a2-c75e70326776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:37:31.559449 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 681/1357] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:37:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:37:31.601845 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c737db9-437f-4da5-8481-9659a4b9a119 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:37:31.603336 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c737db9-437f-4da5-8481-9659a4b9a119 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:37:31.603885 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c737db9-437f-4da5-8481-9659a4b9a119 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:37:31.606187 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 677/1358] 10.222.0.81 () {54 vars in 962 bytes} [Wed Jul 20 22:37:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:37:31.615522 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4a636d-9205-44ff-884e-7ee75a156f28 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:37:31.616233 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4a636d-9205-44ff-884e-7ee75a156f28 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:37:31.616751 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4a636d-9205-44ff-884e-7ee75a156f28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:37:31.720176 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-2d4a636d-9205-44ff-884e-7ee75a156f28 None None] MFA Rules not processed for user `7829324db2574d38b18c749d0559177f`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:37:31.722015 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-2d4a636d-9205-44ff-884e-7ee75a156f28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:37:31.777382 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:37:31.849170 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 682/1359] 10.222.0.81 () {58 vars in 1059 bytes} [Wed Jul 20 22:37:31 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:37:31.858247 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c737db9-437f-4da5-8481-9659a4b9a119 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:37:31.860046 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-8c737db9-437f-4da5-8481-9659a4b9a119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:37:32.030171 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c737db9-437f-4da5-8481-9659a4b9a119 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:37:32.126412 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:37:32.173054 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b41e4c-14e8-4730-a531-560f85f9f01d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7829324db2574d38b18c749d0559177f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:37:32.174276 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b41e4c-14e8-4730-a531-560f85f9f01d service nova] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:37:32.174806 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b41e4c-14e8-4730-a531-560f85f9f01d service nova] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:37:32.175330 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b41e4c-14e8-4730-a531-560f85f9f01d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:37:32.177213 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-26b41e4c-14e8-4730-a531-560f85f9f01d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:37:32.280179 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b41e4c-14e8-4730-a531-560f85f9f01d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:37:32.281769 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b41e4c-14e8-4730-a531-560f85f9f01d service nova] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:37:32.301608 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 678/1360] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:37:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:37:34.551807 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11add52a-e7ee-4198-b46f-32f590852a90 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:37:34.552332 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11add52a-e7ee-4198-b46f-32f590852a90 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:37:34.552332 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11add52a-e7ee-4198-b46f-32f590852a90 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:37:34.553901 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 683/1361] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:37:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:37:34.558901 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5229945-0424-48b7-ae83-395527a82179 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:37:34.559344 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5229945-0424-48b7-ae83-395527a82179 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:37:34.559528 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5229945-0424-48b7-ae83-395527a82179 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:37:34.626785 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-b5229945-0424-48b7-ae83-395527a82179 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:37:34.628060 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-b5229945-0424-48b7-ae83-395527a82179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:37:34.669519 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:37:34.733405 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 679/1362] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:37:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:37:34.777956 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11add52a-e7ee-4198-b46f-32f590852a90 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:37:34.779693 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-11add52a-e7ee-4198-b46f-32f590852a90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:37:34.954260 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11add52a-e7ee-4198-b46f-32f590852a90 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:37:34.965125 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:37:35.119901 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cc48a7-24a6-4987-a606-94a2ba7c2994 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7829324db2574d38b18c749d0559177f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:37:35.120964 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cc48a7-24a6-4987-a606-94a2ba7c2994 service nova] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:37:35.121404 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cc48a7-24a6-4987-a606-94a2ba7c2994 service nova] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:37:35.121835 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cc48a7-24a6-4987-a606-94a2ba7c2994 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:37:35.123848 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-b1cc48a7-24a6-4987-a606-94a2ba7c2994 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:37:35.231071 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cc48a7-24a6-4987-a606-94a2ba7c2994 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:37:35.233199 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cc48a7-24a6-4987-a606-94a2ba7c2994 service nova] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:37:35.254876 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 684/1363] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:37:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 477 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:37:37.774386 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f685cc-fd07-4ccf-b997-09bc7f5d3fe7 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:37:37.774969 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f685cc-fd07-4ccf-b997-09bc7f5d3fe7 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:37:37.774969 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f685cc-fd07-4ccf-b997-09bc7f5d3fe7 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:37:37.776525 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 680/1364] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:37:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:37:37.781621 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f2082b-fec1-4b53-809f-6f2c225a9c10 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:37:37.782085 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f2082b-fec1-4b53-809f-6f2c225a9c10 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:37:37.782185 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f2082b-fec1-4b53-809f-6f2c225a9c10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:37:37.798379 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:37:37.853597 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-07f2082b-fec1-4b53-809f-6f2c225a9c10 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:37:37.854810 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-07f2082b-fec1-4b53-809f-6f2c225a9c10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:37:37.963275 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 685/1365] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:37:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:37:38.016309 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b29eb9-d1ee-4299-8eb9-ec9fcc8871d2 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:37:38.016609 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b29eb9-d1ee-4299-8eb9-ec9fcc8871d2 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:37:38.017214 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b29eb9-d1ee-4299-8eb9-ec9fcc8871d2 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:37:38.019158 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 681/1366] 10.222.0.81 () {54 vars in 962 bytes} [Wed Jul 20 22:37:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:37:38.027890 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f941c14b-7a6d-419e-a76e-1ca1d101e6de None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:37:38.027890 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f941c14b-7a6d-419e-a76e-1ca1d101e6de None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:37:38.027890 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f941c14b-7a6d-419e-a76e-1ca1d101e6de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:37:38.106679 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-f941c14b-7a6d-419e-a76e-1ca1d101e6de None None] MFA Rules not processed for user `7829324db2574d38b18c749d0559177f`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:37:38.108135 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-f941c14b-7a6d-419e-a76e-1ca1d101e6de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:37:38.154645 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:37:38.248587 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 686/1367] 10.222.0.81 () {58 vars in 1059 bytes} [Wed Jul 20 22:37:38 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 22:37:38.257057 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b29eb9-d1ee-4299-8eb9-ec9fcc8871d2 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:37:38.258560 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-38b29eb9-d1ee-4299-8eb9-ec9fcc8871d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:37:38.423450 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b29eb9-d1ee-4299-8eb9-ec9fcc8871d2 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:37:38.444673 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:37:38.541857 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e523ef64-1b74-4af7-a557-26e9ee722d8a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7829324db2574d38b18c749d0559177f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:37:38.542524 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e523ef64-1b74-4af7-a557-26e9ee722d8a service nova] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:37:38.542811 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e523ef64-1b74-4af7-a557-26e9ee722d8a service nova] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:37:38.543119 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e523ef64-1b74-4af7-a557-26e9ee722d8a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:37:38.544763 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-e523ef64-1b74-4af7-a557-26e9ee722d8a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:37:38.644630 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e523ef64-1b74-4af7-a557-26e9ee722d8a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:37:38.646698 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e523ef64-1b74-4af7-a557-26e9ee722d8a service nova] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:37:38.666684 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 682/1368] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:37:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 22:37:39.509836 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ee9e09-f438-44dc-9b4d-f2e8ab4e23d4 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:37:39.510367 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ee9e09-f438-44dc-9b4d-f2e8ab4e23d4 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:37:39.510367 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ee9e09-f438-44dc-9b4d-f2e8ab4e23d4 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:37:39.511895 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 687/1369] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:37:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:37:39.516957 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf0d351-c1da-4746-b7d1-7defc8d6a16f None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:37:39.517391 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf0d351-c1da-4746-b7d1-7defc8d6a16f None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:37:39.517571 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf0d351-c1da-4746-b7d1-7defc8d6a16f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:37:39.548891 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:37:39.588489 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-abf0d351-c1da-4746-b7d1-7defc8d6a16f None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:37:39.589715 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-abf0d351-c1da-4746-b7d1-7defc8d6a16f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:37:39.691626 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 683/1370] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:37:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:37:40.039174 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b67853f-e923-4c05-b9f2-bd7fcefdb392 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:37:40.039368 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b67853f-e923-4c05-b9f2-bd7fcefdb392 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:37:40.039559 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b67853f-e923-4c05-b9f2-bd7fcefdb392 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:37:40.043053 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 688/1371] 10.222.0.81 () {54 vars in 964 bytes} [Wed Jul 20 22:37:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:37:40.046907 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a87af94-ab97-413d-ada5-07d0ae72a8a0 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:37:40.047446 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a87af94-ab97-413d-ada5-07d0ae72a8a0 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:37:40.047807 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a87af94-ab97-413d-ada5-07d0ae72a8a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:37:40.128367 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-3a87af94-ab97-413d-ada5-07d0ae72a8a0 None None] MFA Rules not processed for user `ecf4db289fba4e4a806198c62d31f7c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:37:40.129667 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-3a87af94-ab97-413d-ada5-07d0ae72a8a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:37:40.186504 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:37:40.217302 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 684/1372] 10.222.0.81 () {58 vars in 1061 bytes} [Wed Jul 20 22:37:40 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:37:40.225506 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b67853f-e923-4c05-b9f2-bd7fcefdb392 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:37:40.227519 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-3b67853f-e923-4c05-b9f2-bd7fcefdb392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:37:40.339085 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b67853f-e923-4c05-b9f2-bd7fcefdb392 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:37:40.386240 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:37:40.422078 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b44c0d-7824-4516-ac37-8a6949db75d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecf4db289fba4e4a806198c62d31f7c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:37:40.422763 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b44c0d-7824-4516-ac37-8a6949db75d1 service cinder] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:37:40.423143 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b44c0d-7824-4516-ac37-8a6949db75d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:37:40.423428 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b44c0d-7824-4516-ac37-8a6949db75d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:37:40.425081 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-48b44c0d-7824-4516-ac37-8a6949db75d1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:37:40.539392 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b44c0d-7824-4516-ac37-8a6949db75d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:37:40.540861 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b44c0d-7824-4516-ac37-8a6949db75d1 service cinder] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:37:40.560492 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 689/1373] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:37:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:37:41.008617 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba03b7a-d8e6-43cf-a5a2-865688ed633d None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:37:41.009075 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba03b7a-d8e6-43cf-a5a2-865688ed633d None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:37:41.009360 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba03b7a-d8e6-43cf-a5a2-865688ed633d None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:37:41.010486 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 685/1374] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:37:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:37:41.016010 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9d4e95-1718-4256-8813-a21814a3034e None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:37:41.016301 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9d4e95-1718-4256-8813-a21814a3034e None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:37:41.016650 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9d4e95-1718-4256-8813-a21814a3034e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:37:41.086157 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-ea9d4e95-1718-4256-8813-a21814a3034e None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:37:41.087396 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-ea9d4e95-1718-4256-8813-a21814a3034e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:37:41.109620 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:37:41.190820 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 690/1375] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:37:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:37:41.235445 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba03b7a-d8e6-43cf-a5a2-865688ed633d None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:37:41.236995 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-3ba03b7a-d8e6-43cf-a5a2-865688ed633d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:37:41.382075 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba03b7a-d8e6-43cf-a5a2-865688ed633d None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:37:41.433576 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:37:41.493868 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1123bdac-f349-466b-899f-f9da09644c8f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7829324db2574d38b18c749d0559177f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:37:41.494545 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1123bdac-f349-466b-899f-f9da09644c8f service nova] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:37:41.494827 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1123bdac-f349-466b-899f-f9da09644c8f service nova] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:37:41.495133 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1123bdac-f349-466b-899f-f9da09644c8f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:37:41.496756 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-1123bdac-f349-466b-899f-f9da09644c8f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:37:41.585413 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1123bdac-f349-466b-899f-f9da09644c8f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:37:41.587046 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1123bdac-f349-466b-899f-f9da09644c8f service nova] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:37:41.604731 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 686/1376] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:37:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:37:43.827863 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f67614-d3eb-4ef6-9f35-67422f593277 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:37:43.828432 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f67614-d3eb-4ef6-9f35-67422f593277 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:37:43.828432 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f67614-d3eb-4ef6-9f35-67422f593277 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:37:43.829789 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 691/1377] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:37:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:37:43.835620 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1322197-bae5-43ee-a3ba-070612a96b1b None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:37:43.836032 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1322197-bae5-43ee-a3ba-070612a96b1b None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:37:43.836227 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1322197-bae5-43ee-a3ba-070612a96b1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:37:43.899129 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-b1322197-bae5-43ee-a3ba-070612a96b1b None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:37:43.900329 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-b1322197-bae5-43ee-a3ba-070612a96b1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:37:43.906855 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:37:44.005994 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 687/1378] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:37:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:37:44.049602 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f67614-d3eb-4ef6-9f35-67422f593277 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:37:44.051686 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-13f67614-d3eb-4ef6-9f35-67422f593277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:37:44.256222 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f67614-d3eb-4ef6-9f35-67422f593277 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:37:44.267248 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:37:44.397218 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed7f623-6fc4-44b4-b6a2-db533fe4b131 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7829324db2574d38b18c749d0559177f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:37:44.398002 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed7f623-6fc4-44b4-b6a2-db533fe4b131 service nova] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:37:44.398621 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed7f623-6fc4-44b4-b6a2-db533fe4b131 service nova] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:37:44.399005 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed7f623-6fc4-44b4-b6a2-db533fe4b131 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:37:44.404859 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-3ed7f623-6fc4-44b4-b6a2-db533fe4b131 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:37:44.501440 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed7f623-6fc4-44b4-b6a2-db533fe4b131 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:37:44.503265 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed7f623-6fc4-44b4-b6a2-db533fe4b131 service nova] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:37:44.527189 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 692/1379] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:37:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 478 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:37:46.882876 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb908b3-fb45-497c-bb0e-27d91fdb1074 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:37:46.883482 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb908b3-fb45-497c-bb0e-27d91fdb1074 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:37:46.883482 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb908b3-fb45-497c-bb0e-27d91fdb1074 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:37:46.885237 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 688/1380] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:37:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:37:46.890480 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e563e66a-4dad-44c1-b45d-a0dc0ef347f1 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:37:46.890961 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e563e66a-4dad-44c1-b45d-a0dc0ef347f1 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:37:46.891205 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e563e66a-4dad-44c1-b45d-a0dc0ef347f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:37:46.902121 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:37:46.966846 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-e563e66a-4dad-44c1-b45d-a0dc0ef347f1 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:37:46.968219 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-e563e66a-4dad-44c1-b45d-a0dc0ef347f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:37:47.075201 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 693/1381] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:37:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:37:47.121386 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb908b3-fb45-497c-bb0e-27d91fdb1074 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:37:47.123298 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-1cb908b3-fb45-497c-bb0e-27d91fdb1074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:37:47.282584 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:37:47.294322 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb908b3-fb45-497c-bb0e-27d91fdb1074 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:37:47.413537 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882b9d9d-2449-4739-b499-ce19bec7ed8a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7829324db2574d38b18c749d0559177f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:37:47.414317 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882b9d9d-2449-4739-b499-ce19bec7ed8a service nova] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:37:47.414682 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882b9d9d-2449-4739-b499-ce19bec7ed8a service nova] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:37:47.415062 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882b9d9d-2449-4739-b499-ce19bec7ed8a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:37:47.416770 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-882b9d9d-2449-4739-b499-ce19bec7ed8a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:37:47.509746 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882b9d9d-2449-4739-b499-ce19bec7ed8a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:37:47.511397 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882b9d9d-2449-4739-b499-ce19bec7ed8a service nova] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:37:47.522798 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:37:47.531268 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 689/1382] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:37:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:37:49.851519 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e6949e-4300-41ce-ad32-675b56ba16dc None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:37:49.852207 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e6949e-4300-41ce-ad32-675b56ba16dc None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:37:49.852207 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e6949e-4300-41ce-ad32-675b56ba16dc None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:37:49.853799 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 694/1383] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:37:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:37:49.859120 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99f6828-83b6-4cd4-bef1-dbad9c33aab5 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:37:49.859569 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99f6828-83b6-4cd4-bef1-dbad9c33aab5 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:37:49.859692 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99f6828-83b6-4cd4-bef1-dbad9c33aab5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:37:49.926837 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-a99f6828-83b6-4cd4-bef1-dbad9c33aab5 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:37:49.928114 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-a99f6828-83b6-4cd4-bef1-dbad9c33aab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:37:50.028580 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 690/1384] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:37:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:37:50.071519 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e6949e-4300-41ce-ad32-675b56ba16dc None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:37:50.073062 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-28e6949e-4300-41ce-ad32-675b56ba16dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:37:50.161272 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:37:50.226609 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e6949e-4300-41ce-ad32-675b56ba16dc None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:37:50.362892 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd6532f-0e62-4502-b779-ddbfcea284f1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7829324db2574d38b18c749d0559177f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:37:50.363932 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd6532f-0e62-4502-b779-ddbfcea284f1 service nova] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:37:50.364427 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd6532f-0e62-4502-b779-ddbfcea284f1 service nova] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:37:50.364871 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd6532f-0e62-4502-b779-ddbfcea284f1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:37:50.367485 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-afd6532f-0e62-4502-b779-ddbfcea284f1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:37:50.430801 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:37:50.481775 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd6532f-0e62-4502-b779-ddbfcea284f1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:37:50.484269 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd6532f-0e62-4502-b779-ddbfcea284f1 service nova] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:37:50.512405 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 695/1385] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:37:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:37:52.645350 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e447a20-46e9-43d2-abd5-3d63f5853ed1 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:37:52.645919 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e447a20-46e9-43d2-abd5-3d63f5853ed1 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:37:52.646055 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e447a20-46e9-43d2-abd5-3d63f5853ed1 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:37:52.647600 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 691/1386] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:37:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:37:52.652926 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c3c7be-8bde-458d-8013-81ecec8b601b None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:37:52.653346 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c3c7be-8bde-458d-8013-81ecec8b601b None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:37:52.653640 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c3c7be-8bde-458d-8013-81ecec8b601b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:37:52.718747 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-18c3c7be-8bde-458d-8013-81ecec8b601b None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:37:52.720009 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-18c3c7be-8bde-458d-8013-81ecec8b601b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:37:52.825376 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 696/1387] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:37:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:37:52.871787 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e447a20-46e9-43d2-abd5-3d63f5853ed1 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:37:52.873339 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-2e447a20-46e9-43d2-abd5-3d63f5853ed1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:37:52.951107 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:37:53.029478 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e447a20-46e9-43d2-abd5-3d63f5853ed1 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:37:53.158012 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56cab8c-b761-442c-94db-a8be774de0b2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7829324db2574d38b18c749d0559177f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:37:53.158675 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56cab8c-b761-442c-94db-a8be774de0b2 service nova] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:37:53.158956 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56cab8c-b761-442c-94db-a8be774de0b2 service nova] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:37:53.159261 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56cab8c-b761-442c-94db-a8be774de0b2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:37:53.160931 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-f56cab8c-b761-442c-94db-a8be774de0b2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:37:53.202438 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:37:53.267840 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56cab8c-b761-442c-94db-a8be774de0b2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:37:53.270098 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56cab8c-b761-442c-94db-a8be774de0b2 service nova] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:37:53.294357 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 692/1388] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:37:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:37:55.536233 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be98185a-d301-41f0-87b7-c04bd5a442f1 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:37:55.537021 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be98185a-d301-41f0-87b7-c04bd5a442f1 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:37:55.537021 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be98185a-d301-41f0-87b7-c04bd5a442f1 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:37:55.538322 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 697/1389] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:37:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:37:55.543913 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64978267-e72e-4611-84d5-2ee4c27bca9b None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:37:55.544352 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64978267-e72e-4611-84d5-2ee4c27bca9b None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:37:55.544478 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64978267-e72e-4611-84d5-2ee4c27bca9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:37:55.609272 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-64978267-e72e-4611-84d5-2ee4c27bca9b None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:37:55.610477 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-64978267-e72e-4611-84d5-2ee4c27bca9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:37:55.706528 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 693/1390] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:37:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:37:55.756260 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be98185a-d301-41f0-87b7-c04bd5a442f1 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:37:55.757778 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-be98185a-d301-41f0-87b7-c04bd5a442f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:37:55.790661 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:37:55.905214 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be98185a-d301-41f0-87b7-c04bd5a442f1 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:37:56.007061 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:37:56.018591 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0052ba-4b18-47d1-95c2-bc476612379b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7829324db2574d38b18c749d0559177f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:37:56.019774 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0052ba-4b18-47d1-95c2-bc476612379b service nova] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:37:56.020077 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0052ba-4b18-47d1-95c2-bc476612379b service nova] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:37:56.020337 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0052ba-4b18-47d1-95c2-bc476612379b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:37:56.021951 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-cc0052ba-4b18-47d1-95c2-bc476612379b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:37:56.032587 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e928caf-6625-4d7e-a237-39794d7fc91d None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:37:56.032908 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e928caf-6625-4d7e-a237-39794d7fc91d None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:37:56.033201 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e928caf-6625-4d7e-a237-39794d7fc91d None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:37:56.034586 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 694/1391] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:37:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 22:37:56.040063 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4878c338-f2fd-4a22-85e5-d0c6aca68852 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:37:56.040385 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4878c338-f2fd-4a22-85e5-d0c6aca68852 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:37:56.040649 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4878c338-f2fd-4a22-85e5-d0c6aca68852 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:37:56.055246 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:37:56.111345 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-4878c338-f2fd-4a22-85e5-d0c6aca68852 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:37:56.112591 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-4878c338-f2fd-4a22-85e5-d0c6aca68852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:37:56.226075 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 695/1392] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:37:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 22:37:56.255135 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0052ba-4b18-47d1-95c2-bc476612379b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:37:56.256197 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0052ba-4b18-47d1-95c2-bc476612379b service nova] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:37:56.275366 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 698/1393] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:37:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 519 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:37:56.283681 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c764d8-1ce4-444b-a641-89458af0dd74 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:37:56.284032 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c764d8-1ce4-444b-a641-89458af0dd74 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:37:56.292102 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c764d8-1ce4-444b-a641-89458af0dd74 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:37:56.292102 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 696/1394] 10.222.0.81 () {54 vars in 966 bytes} [Wed Jul 20 22:37:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:37:56.292297 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20abcb8c-c3a9-4e69-9616-8f498739cf54 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:37:56.292297 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20abcb8c-c3a9-4e69-9616-8f498739cf54 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:37:56.292297 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20abcb8c-c3a9-4e69-9616-8f498739cf54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:37:56.365039 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-20abcb8c-c3a9-4e69-9616-8f498739cf54 None None] MFA Rules not processed for user `9c44640ea0014426a9523bb1c457778b`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:37:56.366824 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-20abcb8c-c3a9-4e69-9616-8f498739cf54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:37:56.424536 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:37:56.449483 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 699/1395] 10.222.0.81 () {58 vars in 1063 bytes} [Wed Jul 20 22:37:56 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:37:56.457173 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c764d8-1ce4-444b-a641-89458af0dd74 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:37:56.458735 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-65c764d8-1ce4-444b-a641-89458af0dd74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:37:56.561466 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:37:56.569145 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c764d8-1ce4-444b-a641-89458af0dd74 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:37:56.649927 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081cbb7d-56e8-4d62-9f05-3a01b446435a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c44640ea0014426a9523bb1c457778b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:37:56.650506 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081cbb7d-56e8-4d62-9f05-3a01b446435a service glance] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:37:56.650790 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081cbb7d-56e8-4d62-9f05-3a01b446435a service glance] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:37:56.651100 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081cbb7d-56e8-4d62-9f05-3a01b446435a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:37:56.652769 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-081cbb7d-56e8-4d62-9f05-3a01b446435a service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:37:56.762978 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081cbb7d-56e8-4d62-9f05-3a01b446435a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:37:56.764451 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081cbb7d-56e8-4d62-9f05-3a01b446435a service glance] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:37:56.783361 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 697/1396] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:37:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:37:57.726097 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7033b3-642a-4330-ae27-e50c18537789 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:37:57.726733 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7033b3-642a-4330-ae27-e50c18537789 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:37:57.726733 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7033b3-642a-4330-ae27-e50c18537789 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:37:57.794170 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-2e7033b3-642a-4330-ae27-e50c18537789 None None] MFA Rules not processed for user `90915e1264324f16bc7bae7e46e0c97b`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:37:57.795469 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-2e7033b3-642a-4330-ae27-e50c18537789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:37:57.883578 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 700/1397] 10.222.0.81 () {58 vars in 1009 bytes} [Wed Jul 20 22:37:57 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:37:57.926407 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaea4c8-b9fd-4d3f-aa6b-8761d73b804c None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:37:57.929266 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaea4c8-b9fd-4d3f-aa6b-8761d73b804c None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:37:57.929266 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaea4c8-b9fd-4d3f-aa6b-8761d73b804c None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:37:57.929266 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 698/1398] 10.222.0.81 () {54 vars in 943 bytes} [Wed Jul 20 22:37:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:37:57.935263 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6d2335-c28e-4409-a12b-ce3882c3dc76 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:37:57.935560 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6d2335-c28e-4409-a12b-ce3882c3dc76 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:37:57.935847 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6d2335-c28e-4409-a12b-ce3882c3dc76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:37:58.015496 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-2c6d2335-c28e-4409-a12b-ce3882c3dc76 None None] MFA Rules not processed for user `70e97436a08543a180225561ef8bc817`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:37:58.016610 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-2c6d2335-c28e-4409-a12b-ce3882c3dc76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:37:58.029744 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:37:58.122884 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 701/1399] 10.222.0.81 () {58 vars in 1040 bytes} [Wed Jul 20 22:37:57 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:37:58.133097 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaea4c8-b9fd-4d3f-aa6b-8761d73b804c None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:37:58.134749 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-cfaea4c8-b9fd-4d3f-aa6b-8761d73b804c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:37:58.158664 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:37:58.271116 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaea4c8-b9fd-4d3f-aa6b-8761d73b804c None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:37:58.382974 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afaccd0-5d88-4683-8c73-c45b885defb4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70e97436a08543a180225561ef8bc817', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:37:58.383671 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afaccd0-5d88-4683-8c73-c45b885defb4 service swift] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:37:58.384064 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afaccd0-5d88-4683-8c73-c45b885defb4 service swift] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:37:58.384274 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afaccd0-5d88-4683-8c73-c45b885defb4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:37:58.386381 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-5afaccd0-5d88-4683-8c73-c45b885defb4 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:37:58.402256 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:37:58.480698 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afaccd0-5d88-4683-8c73-c45b885defb4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:37:58.482308 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afaccd0-5d88-4683-8c73-c45b885defb4 service swift] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:37:58.491475 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 699/1400] 10.222.0.81 () {60 vars in 1384 bytes} [Wed Jul 20 22:37:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 22:37:58.582845 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3090642f-6ae4-45db-8323-ed6702fd01a4 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:37:58.583165 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3090642f-6ae4-45db-8323-ed6702fd01a4 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:37:58.583444 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3090642f-6ae4-45db-8323-ed6702fd01a4 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:37:58.584824 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 702/1401] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:37:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:37:58.590061 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a990c94c-8892-46a7-bb41-21cad16055a2 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:37:58.590356 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a990c94c-8892-46a7-bb41-21cad16055a2 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:37:58.590635 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a990c94c-8892-46a7-bb41-21cad16055a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:37:58.654474 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-a990c94c-8892-46a7-bb41-21cad16055a2 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:37:58.655700 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-a990c94c-8892-46a7-bb41-21cad16055a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:37:58.750435 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:37:58.759314 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 700/1402] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:37:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:37:58.797975 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3090642f-6ae4-45db-8323-ed6702fd01a4 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:37:58.801596 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-3090642f-6ae4-45db-8323-ed6702fd01a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:37:58.938622 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:37:58.959166 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3090642f-6ae4-45db-8323-ed6702fd01a4 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:37:59.088469 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf9a97d-0ecc-4b58-9b07-b240a5357253 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7829324db2574d38b18c749d0559177f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:37:59.088821 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf9a97d-0ecc-4b58-9b07-b240a5357253 service nova] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:37:59.089439 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf9a97d-0ecc-4b58-9b07-b240a5357253 service nova] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:37:59.089557 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf9a97d-0ecc-4b58-9b07-b240a5357253 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:37:59.091718 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-9bf9a97d-0ecc-4b58-9b07-b240a5357253 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:37:59.204854 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf9a97d-0ecc-4b58-9b07-b240a5357253 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:37:59.207107 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf9a97d-0ecc-4b58-9b07-b240a5357253 service nova] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:37:59.212256 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:37:59.228554 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 703/1403] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:37:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:38:01.451396 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c806d088-e646-47e7-8fec-d9b997d12196 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:38:01.451922 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c806d088-e646-47e7-8fec-d9b997d12196 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:38:01.451922 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c806d088-e646-47e7-8fec-d9b997d12196 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:38:01.453429 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 701/1404] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:38:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:38:01.459393 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eba697-dd06-4d10-8a32-0518ef486ea3 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:38:01.459893 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eba697-dd06-4d10-8a32-0518ef486ea3 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:38:01.459978 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eba697-dd06-4d10-8a32-0518ef486ea3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:38:01.528099 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-49eba697-dd06-4d10-8a32-0518ef486ea3 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:38:01.529326 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-49eba697-dd06-4d10-8a32-0518ef486ea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:38:01.637825 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 704/1405] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:38:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:38:01.685626 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c806d088-e646-47e7-8fec-d9b997d12196 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:38:01.686172 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-c806d088-e646-47e7-8fec-d9b997d12196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:38:01.858927 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c806d088-e646-47e7-8fec-d9b997d12196 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:38:01.951860 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:38:01.994995 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0157a57-742e-4b8e-ba98-64fff5274045 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7829324db2574d38b18c749d0559177f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:38:01.995715 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0157a57-742e-4b8e-ba98-64fff5274045 service nova] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:38:01.996004 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0157a57-742e-4b8e-ba98-64fff5274045 service nova] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:38:01.996290 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0157a57-742e-4b8e-ba98-64fff5274045 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:38:01.998222 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-f0157a57-742e-4b8e-ba98-64fff5274045 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:38:02.104339 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0157a57-742e-4b8e-ba98-64fff5274045 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:38:02.105976 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0157a57-742e-4b8e-ba98-64fff5274045 service nova] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:38:02.124035 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 702/1406] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:38:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:38:04.706729 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daa95b8-0839-4e3f-af46-18bb3108e5ab None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:38:04.707360 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daa95b8-0839-4e3f-af46-18bb3108e5ab None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:38:04.707360 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daa95b8-0839-4e3f-af46-18bb3108e5ab None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:38:04.709109 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 705/1407] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:38:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:38:04.714472 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76c03ea-32d9-4357-bbff-8be63950fb1e None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:38:04.714903 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76c03ea-32d9-4357-bbff-8be63950fb1e None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:38:04.715123 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76c03ea-32d9-4357-bbff-8be63950fb1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:38:04.787958 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-f76c03ea-32d9-4357-bbff-8be63950fb1e None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:38:04.789189 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-f76c03ea-32d9-4357-bbff-8be63950fb1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:38:04.831482 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:38:04.898589 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 703/1408] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:38:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:38:04.945025 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daa95b8-0839-4e3f-af46-18bb3108e5ab None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:38:04.946722 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-3daa95b8-0839-4e3f-af46-18bb3108e5ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:38:05.031811 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:38:05.114755 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daa95b8-0839-4e3f-af46-18bb3108e5ab None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:38:05.255551 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0201a138-0d3d-4ea7-8e67-1307466dfde6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7829324db2574d38b18c749d0559177f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:38:05.256207 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0201a138-0d3d-4ea7-8e67-1307466dfde6 service nova] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:38:05.256495 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0201a138-0d3d-4ea7-8e67-1307466dfde6 service nova] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:38:05.256772 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0201a138-0d3d-4ea7-8e67-1307466dfde6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:38:05.258402 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-0201a138-0d3d-4ea7-8e67-1307466dfde6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:38:05.291551 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:38:05.357004 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0201a138-0d3d-4ea7-8e67-1307466dfde6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:38:05.358607 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0201a138-0d3d-4ea7-8e67-1307466dfde6 service nova] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:38:05.378576 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 706/1409] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:38:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:38:14.896790 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=27 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:38:15.375649 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=32 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:38:22.344329 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d716b170-34fe-487a-96d6-88f60196f4ef None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:38:22.344876 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d716b170-34fe-487a-96d6-88f60196f4ef None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:38:22.344876 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d716b170-34fe-487a-96d6-88f60196f4ef None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:38:22.346277 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 704/1410] 10.222.0.81 () {54 vars in 918 bytes} [Wed Jul 20 22:38:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:38:22.352390 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64183b39-9b7b-4a80-9e62-1e86988b74a1 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:38:22.352822 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64183b39-9b7b-4a80-9e62-1e86988b74a1 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:38:22.352935 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64183b39-9b7b-4a80-9e62-1e86988b74a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:38:22.431241 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-64183b39-9b7b-4a80-9e62-1e86988b74a1 None None] MFA Rules not processed for user `d5500cebdd9d4f8e9aecb45d6c955878`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:38:22.432445 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-64183b39-9b7b-4a80-9e62-1e86988b74a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:38:22.537280 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 707/1411] 10.222.0.81 () {58 vars in 1015 bytes} [Wed Jul 20 22:38:22 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:38:22.551518 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8072f7b-a798-4e4f-8883-300da158cf25 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:38:22.551808 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8072f7b-a798-4e4f-8883-300da158cf25 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:38:22.552086 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8072f7b-a798-4e4f-8883-300da158cf25 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:38:22.553627 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 705/1412] 10.222.0.81 () {54 vars in 918 bytes} [Wed Jul 20 22:38:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:38:22.558168 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07565576-ea65-4af5-856e-3e5fa834c0e1 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:38:22.558455 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07565576-ea65-4af5-856e-3e5fa834c0e1 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:38:22.558734 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07565576-ea65-4af5-856e-3e5fa834c0e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:38:22.629749 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:38:22.632579 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-07565576-ea65-4af5-856e-3e5fa834c0e1 None None] MFA Rules not processed for user `7829324db2574d38b18c749d0559177f`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:38:22.633713 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-07565576-ea65-4af5-856e-3e5fa834c0e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:38:22.753994 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 708/1413] 10.222.0.81 () {58 vars in 1015 bytes} [Wed Jul 20 22:38:22 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 22:38:25.250617 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47bd46c-924f-4ab7-a0bd-711c696d2c63 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:38:25.251133 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47bd46c-924f-4ab7-a0bd-711c696d2c63 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:38:25.251247 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47bd46c-924f-4ab7-a0bd-711c696d2c63 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:38:25.252573 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 706/1414] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 22:38:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:38:25.257938 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b89efa8-a635-4204-a1a4-77323f8516e2 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:38:25.258371 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b89efa8-a635-4204-a1a4-77323f8516e2 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:38:25.258503 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b89efa8-a635-4204-a1a4-77323f8516e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:38:25.322877 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-9b89efa8-a635-4204-a1a4-77323f8516e2 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:38:25.324188 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-9b89efa8-a635-4204-a1a4-77323f8516e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:38:25.379770 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:38:25.429729 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 709/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 22:38:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:38:25.473459 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47bd46c-924f-4ab7-a0bd-711c696d2c63 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:38:25.474987 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-a47bd46c-924f-4ab7-a0bd-711c696d2c63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:38:25.617964 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47bd46c-924f-4ab7-a0bd-711c696d2c63 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:38:25.688468 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:38:25.728457 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58605f3-5fe8-43bd-a42b-bbe4c845705b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7829324db2574d38b18c749d0559177f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:38:25.729044 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58605f3-5fe8-43bd-a42b-bbe4c845705b service nova] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:38:25.729335 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58605f3-5fe8-43bd-a42b-bbe4c845705b service nova] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:38:25.729617 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58605f3-5fe8-43bd-a42b-bbe4c845705b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:38:25.731293 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-e58605f3-5fe8-43bd-a42b-bbe4c845705b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:38:25.819656 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58605f3-5fe8-43bd-a42b-bbe4c845705b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:38:25.821279 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58605f3-5fe8-43bd-a42b-bbe4c845705b service nova] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:38:25.838964 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 707/1416] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:38:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:38:26.768007 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13ddb67-f8cc-429d-a405-e3a5d0562a17 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:38:26.768580 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13ddb67-f8cc-429d-a405-e3a5d0562a17 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:38:26.768580 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13ddb67-f8cc-429d-a405-e3a5d0562a17 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:38:26.769935 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 710/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 22:38:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:38:26.774853 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d75856-467b-4625-8f9d-c54f7223bfe2 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:38:26.775350 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d75856-467b-4625-8f9d-c54f7223bfe2 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:38:26.775494 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d75856-467b-4625-8f9d-c54f7223bfe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:38:26.849280 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-30d75856-467b-4625-8f9d-c54f7223bfe2 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:38:26.850480 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-30d75856-467b-4625-8f9d-c54f7223bfe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:38:26.881158 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:38:26.950898 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 708/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 22:38:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:38:27.101266 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13ddb67-f8cc-429d-a405-e3a5d0562a17 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:38:27.102830 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-f13ddb67-f8cc-429d-a405-e3a5d0562a17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:38:27.248376 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13ddb67-f8cc-429d-a405-e3a5d0562a17 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:38:27.288922 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:38:27.360417 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27d708f-6370-462b-8ac0-02fd2d39ea25 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7829324db2574d38b18c749d0559177f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:38:27.360988 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27d708f-6370-462b-8ac0-02fd2d39ea25 service nova] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:38:27.361275 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27d708f-6370-462b-8ac0-02fd2d39ea25 service nova] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:38:27.361555 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27d708f-6370-462b-8ac0-02fd2d39ea25 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:38:27.363269 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-a27d708f-6370-462b-8ac0-02fd2d39ea25 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:38:27.450091 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27d708f-6370-462b-8ac0-02fd2d39ea25 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:38:27.451724 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27d708f-6370-462b-8ac0-02fd2d39ea25 service nova] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:38:27.470069 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 711/1419] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:38:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:38:28.352771 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbcf0c3-d382-4c29-80e6-37685f3f9f3b None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:38:28.353310 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbcf0c3-d382-4c29-80e6-37685f3f9f3b None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:38:28.353310 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbcf0c3-d382-4c29-80e6-37685f3f9f3b None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:38:28.354712 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 709/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 22:38:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:38:28.359811 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33889e32-d0c6-4ecf-b16d-aa261fa77ddd None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:38:28.363376 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33889e32-d0c6-4ecf-b16d-aa261fa77ddd None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:38:28.363376 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33889e32-d0c6-4ecf-b16d-aa261fa77ddd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:38:28.413976 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:38:28.436763 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-33889e32-d0c6-4ecf-b16d-aa261fa77ddd None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:38:28.438021 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-33889e32-d0c6-4ecf-b16d-aa261fa77ddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:38:28.533219 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 712/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 22:38:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:38:28.685437 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbcf0c3-d382-4c29-80e6-37685f3f9f3b None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:38:28.687124 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-adbcf0c3-d382-4c29-80e6-37685f3f9f3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:38:28.836047 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbcf0c3-d382-4c29-80e6-37685f3f9f3b None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:38:28.848648 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:38:28.948127 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb02773f-ae8a-40d8-8453-cc292f3dd4ff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7829324db2574d38b18c749d0559177f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:38:28.948852 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb02773f-ae8a-40d8-8453-cc292f3dd4ff service nova] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:38:28.949140 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb02773f-ae8a-40d8-8453-cc292f3dd4ff service nova] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:38:28.949416 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb02773f-ae8a-40d8-8453-cc292f3dd4ff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:38:28.952265 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-eb02773f-ae8a-40d8-8453-cc292f3dd4ff service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:38:29.044585 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb02773f-ae8a-40d8-8453-cc292f3dd4ff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:38:29.046200 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb02773f-ae8a-40d8-8453-cc292f3dd4ff service nova] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:38:29.064499 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 710/1422] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:38:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:38:29.926496 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6619c55e-5d42-4cf5-8c7b-2b6a459f813e None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:38:29.927117 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6619c55e-5d42-4cf5-8c7b-2b6a459f813e None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:38:29.927117 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6619c55e-5d42-4cf5-8c7b-2b6a459f813e None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:38:29.928469 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 713/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 22:38:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:38:29.933562 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a876f9-aed2-4714-a62b-1f5abd307ce6 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:38:29.934049 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a876f9-aed2-4714-a62b-1f5abd307ce6 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:38:29.934164 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a876f9-aed2-4714-a62b-1f5abd307ce6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:38:29.953418 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:38:30.011527 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-d0a876f9-aed2-4714-a62b-1f5abd307ce6 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:38:30.012751 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-d0a876f9-aed2-4714-a62b-1f5abd307ce6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:38:30.111095 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 711/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 22:38:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:38:30.262621 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6619c55e-5d42-4cf5-8c7b-2b6a459f813e None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:38:30.264246 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-6619c55e-5d42-4cf5-8c7b-2b6a459f813e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:38:30.373537 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:38:30.409310 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6619c55e-5d42-4cf5-8c7b-2b6a459f813e None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:38:30.521312 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1208600-affd-49e3-9453-c79dff749c3c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7829324db2574d38b18c749d0559177f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:38:30.521867 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1208600-affd-49e3-9453-c79dff749c3c service nova] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:38:30.522146 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1208600-affd-49e3-9453-c79dff749c3c service nova] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:38:30.522418 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1208600-affd-49e3-9453-c79dff749c3c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:38:30.524107 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-f1208600-affd-49e3-9453-c79dff749c3c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:38:30.600443 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:38:30.613502 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1208600-affd-49e3-9453-c79dff749c3c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:38:30.615134 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1208600-affd-49e3-9453-c79dff749c3c service nova] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:38:30.633861 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 714/1425] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:38:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:38:31.498344 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428dca07-8dee-448a-867c-086d48816269 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:38:31.498876 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428dca07-8dee-448a-867c-086d48816269 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:38:31.498876 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428dca07-8dee-448a-867c-086d48816269 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:38:31.500220 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 712/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 22:38:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:38:31.505934 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582b550c-bdd2-490e-95a7-61e07acf077c None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:38:31.506382 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582b550c-bdd2-490e-95a7-61e07acf077c None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:38:31.506489 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582b550c-bdd2-490e-95a7-61e07acf077c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:38:31.580543 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-582b550c-bdd2-490e-95a7-61e07acf077c None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:38:31.582414 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-582b550c-bdd2-490e-95a7-61e07acf077c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:38:31.678708 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 715/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 22:38:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:38:31.837168 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428dca07-8dee-448a-867c-086d48816269 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:38:31.838729 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-428dca07-8dee-448a-867c-086d48816269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:38:31.924089 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:38:31.984149 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428dca07-8dee-448a-867c-086d48816269 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:38:32.100583 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef1d9dd-247c-49a0-b9f7-101c97d112d9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7829324db2574d38b18c749d0559177f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:38:32.101206 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef1d9dd-247c-49a0-b9f7-101c97d112d9 service nova] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:38:32.101493 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef1d9dd-247c-49a0-b9f7-101c97d112d9 service nova] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:38:32.101772 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef1d9dd-247c-49a0-b9f7-101c97d112d9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:38:32.103631 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-7ef1d9dd-247c-49a0-b9f7-101c97d112d9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:38:32.153008 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:38:32.194054 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef1d9dd-247c-49a0-b9f7-101c97d112d9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:38:32.196049 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef1d9dd-247c-49a0-b9f7-101c97d112d9 service nova] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:38:32.215092 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 713/1428] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:38:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:38:33.116086 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0f433b-f6b4-487e-a931-d5d871de2103 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:38:33.116651 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0f433b-f6b4-487e-a931-d5d871de2103 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:38:33.116651 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0f433b-f6b4-487e-a931-d5d871de2103 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:38:33.118025 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 716/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 22:38:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:38:33.122964 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ce1d84-8476-4c1b-9056-a591ed45bf8c None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:38:33.123399 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ce1d84-8476-4c1b-9056-a591ed45bf8c None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:38:33.123555 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ce1d84-8476-4c1b-9056-a591ed45bf8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:38:33.200292 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-60ce1d84-8476-4c1b-9056-a591ed45bf8c None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:38:33.201598 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-60ce1d84-8476-4c1b-9056-a591ed45bf8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:38:33.297480 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 714/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 22:38:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:38:33.446503 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0f433b-f6b4-487e-a931-d5d871de2103 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:38:33.448101 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-3a0f433b-f6b4-487e-a931-d5d871de2103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:38:33.503914 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:38:33.604038 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0f433b-f6b4-487e-a931-d5d871de2103 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:38:33.719597 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a124e7e7-201b-4186-8164-81638bda30b5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7829324db2574d38b18c749d0559177f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:38:33.720247 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a124e7e7-201b-4186-8164-81638bda30b5 service nova] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:38:33.720534 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a124e7e7-201b-4186-8164-81638bda30b5 service nova] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:38:33.720814 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a124e7e7-201b-4186-8164-81638bda30b5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:38:33.722502 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-a124e7e7-201b-4186-8164-81638bda30b5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:38:33.729454 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:38:33.812921 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a124e7e7-201b-4186-8164-81638bda30b5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:38:33.814527 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a124e7e7-201b-4186-8164-81638bda30b5 service nova] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:38:33.834001 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 717/1431] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:38:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:38:34.727215 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed35f4c5-b697-4aa3-aa51-51cded9669df None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:38:34.727800 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed35f4c5-b697-4aa3-aa51-51cded9669df None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:38:34.727800 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed35f4c5-b697-4aa3-aa51-51cded9669df None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:38:34.729235 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 715/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 22:38:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:38:34.734078 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c01ba0f-6cd2-4610-91d7-e45b69d2871b None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:38:34.734516 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c01ba0f-6cd2-4610-91d7-e45b69d2871b None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:38:34.734626 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c01ba0f-6cd2-4610-91d7-e45b69d2871b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:38:34.810603 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-6c01ba0f-6cd2-4610-91d7-e45b69d2871b None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:38:34.811900 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-6c01ba0f-6cd2-4610-91d7-e45b69d2871b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:38:34.880602 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:38:34.910221 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 718/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 22:38:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:38:35.060587 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed35f4c5-b697-4aa3-aa51-51cded9669df None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:38:35.062164 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-ed35f4c5-b697-4aa3-aa51-51cded9669df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:38:35.080103 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:38:35.213510 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed35f4c5-b697-4aa3-aa51-51cded9669df None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:38:35.298956 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:38:35.324263 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f59ebc-5cd7-4bbf-a5b8-c836dbca7acd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7829324db2574d38b18c749d0559177f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:38:35.324862 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f59ebc-5cd7-4bbf-a5b8-c836dbca7acd service nova] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:38:35.325147 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f59ebc-5cd7-4bbf-a5b8-c836dbca7acd service nova] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:38:35.325424 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f59ebc-5cd7-4bbf-a5b8-c836dbca7acd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:38:35.327144 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-c9f59ebc-5cd7-4bbf-a5b8-c836dbca7acd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:38:35.415321 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f59ebc-5cd7-4bbf-a5b8-c836dbca7acd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:38:35.416930 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f59ebc-5cd7-4bbf-a5b8-c836dbca7acd service nova] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:38:35.435187 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 716/1434] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:38:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:38:36.297692 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68255492-d815-4892-8ad5-fe645049b5ef None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:38:36.298326 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68255492-d815-4892-8ad5-fe645049b5ef None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:38:36.298326 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68255492-d815-4892-8ad5-fe645049b5ef None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:38:36.299849 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 719/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 22:38:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:38:36.304833 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6e0b14-629b-4e18-823e-173258f76dc7 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:38:36.305244 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6e0b14-629b-4e18-823e-173258f76dc7 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:38:36.305397 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6e0b14-629b-4e18-823e-173258f76dc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:38:36.379327 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-7d6e0b14-629b-4e18-823e-173258f76dc7 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:38:36.380554 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-7d6e0b14-629b-4e18-823e-173258f76dc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:38:36.422305 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:38:36.479672 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 717/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 22:38:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:38:36.632096 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68255492-d815-4892-8ad5-fe645049b5ef None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:38:36.633597 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-68255492-d815-4892-8ad5-fe645049b5ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:38:36.780762 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68255492-d815-4892-8ad5-fe645049b5ef None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:38:36.836561 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:38:36.896175 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfa3b95-2544-4cc8-9dd2-398d7b4b06ce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7829324db2574d38b18c749d0559177f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:38:36.896825 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfa3b95-2544-4cc8-9dd2-398d7b4b06ce service nova] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:38:36.897106 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfa3b95-2544-4cc8-9dd2-398d7b4b06ce service nova] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:38:36.897380 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfa3b95-2544-4cc8-9dd2-398d7b4b06ce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:38:36.899033 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-dbfa3b95-2544-4cc8-9dd2-398d7b4b06ce service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:38:36.989281 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfa3b95-2544-4cc8-9dd2-398d7b4b06ce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:38:36.990867 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfa3b95-2544-4cc8-9dd2-398d7b4b06ce service nova] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:38:37.008420 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 720/1437] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:38:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:38:37.884606 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412e7cc1-66c8-42e2-a615-cf5b54756272 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:38:37.885174 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412e7cc1-66c8-42e2-a615-cf5b54756272 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:38:37.885174 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412e7cc1-66c8-42e2-a615-cf5b54756272 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:38:37.886616 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 718/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 22:38:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:38:37.891882 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da676646-7a99-444f-ac2a-3b0f68949edc None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:38:37.892337 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da676646-7a99-444f-ac2a-3b0f68949edc None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:38:37.892444 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da676646-7a99-444f-ac2a-3b0f68949edc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:38:37.967037 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-da676646-7a99-444f-ac2a-3b0f68949edc None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:38:37.968265 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-da676646-7a99-444f-ac2a-3b0f68949edc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:38:37.970272 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:38:38.065360 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 721/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 22:38:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:38:38.223846 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412e7cc1-66c8-42e2-a615-cf5b54756272 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:38:38.225416 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-412e7cc1-66c8-42e2-a615-cf5b54756272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:38:38.380309 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412e7cc1-66c8-42e2-a615-cf5b54756272 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:38:38.410873 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:38:38.498431 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f9bcf0-6922-45ae-86a9-54819a7ab875 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7829324db2574d38b18c749d0559177f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:38:38.499124 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f9bcf0-6922-45ae-86a9-54819a7ab875 service nova] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:38:38.499418 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f9bcf0-6922-45ae-86a9-54819a7ab875 service nova] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:38:38.499701 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f9bcf0-6922-45ae-86a9-54819a7ab875 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:38:38.501356 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-04f9bcf0-6922-45ae-86a9-54819a7ab875 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:38:38.591442 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f9bcf0-6922-45ae-86a9-54819a7ab875 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:38:38.593098 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f9bcf0-6922-45ae-86a9-54819a7ab875 service nova] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:38:38.610806 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 719/1440] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:38:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:38:40.579829 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51333aad-ecaf-4f0a-9a09-a70bfdb7fdf7 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:38:40.580379 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51333aad-ecaf-4f0a-9a09-a70bfdb7fdf7 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:38:40.580379 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51333aad-ecaf-4f0a-9a09-a70bfdb7fdf7 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:38:40.581770 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 722/1441] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 22:38:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:38:40.587287 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9642d98-be57-4f27-8fe2-18ada83d293b None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:38:40.587725 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9642d98-be57-4f27-8fe2-18ada83d293b None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:38:40.587890 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9642d98-be57-4f27-8fe2-18ada83d293b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:38:40.633872 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:38:40.659412 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-c9642d98-be57-4f27-8fe2-18ada83d293b None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:38:40.661140 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-c9642d98-be57-4f27-8fe2-18ada83d293b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:38:40.758202 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 720/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 22:38:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:38:40.797934 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51333aad-ecaf-4f0a-9a09-a70bfdb7fdf7 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:38:40.799541 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-51333aad-ecaf-4f0a-9a09-a70bfdb7fdf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:38:40.905038 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51333aad-ecaf-4f0a-9a09-a70bfdb7fdf7 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:38:40.937245 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:38:40.985208 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdcb43e-7dad-4c79-b684-f99270f94751 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:38:40.985860 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdcb43e-7dad-4c79-b684-f99270f94751 service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:38:40.986148 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdcb43e-7dad-4c79-b684-f99270f94751 service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:38:40.986431 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdcb43e-7dad-4c79-b684-f99270f94751 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:38:40.988108 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-fcdcb43e-7dad-4c79-b684-f99270f94751 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:38:41.095681 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdcb43e-7dad-4c79-b684-f99270f94751 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:38:41.097128 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdcb43e-7dad-4c79-b684-f99270f94751 service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:38:41.115981 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 723/1443] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:38:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:38:43.358268 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae42d6ec-b655-4e1e-a166-1b82eb028c78 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:38:43.358823 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae42d6ec-b655-4e1e-a166-1b82eb028c78 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:38:43.358823 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae42d6ec-b655-4e1e-a166-1b82eb028c78 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:38:43.360230 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 721/1444] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 22:38:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:38:43.365537 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb77246-d70b-4233-804d-e5b10335ca33 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:38:43.365929 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb77246-d70b-4233-804d-e5b10335ca33 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:38:43.366089 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb77246-d70b-4233-804d-e5b10335ca33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:38:43.432341 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-8cb77246-d70b-4233-804d-e5b10335ca33 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:38:43.433548 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-8cb77246-d70b-4233-804d-e5b10335ca33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:38:43.446767 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:38:43.530493 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 724/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 22:38:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:38:43.569326 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae42d6ec-b655-4e1e-a166-1b82eb028c78 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:38:43.570836 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-ae42d6ec-b655-4e1e-a166-1b82eb028c78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:38:43.671605 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae42d6ec-b655-4e1e-a166-1b82eb028c78 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:38:43.683185 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:38:43.745345 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281c9139-f2e7-4f96-a457-bcc7d8195b15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:38:43.745965 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281c9139-f2e7-4f96-a457-bcc7d8195b15 service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:38:43.746328 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281c9139-f2e7-4f96-a457-bcc7d8195b15 service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:38:43.746604 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281c9139-f2e7-4f96-a457-bcc7d8195b15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:38:43.748259 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-281c9139-f2e7-4f96-a457-bcc7d8195b15 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:38:43.854085 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281c9139-f2e7-4f96-a457-bcc7d8195b15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:38:43.855557 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281c9139-f2e7-4f96-a457-bcc7d8195b15 service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:38:43.873958 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 722/1446] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:38:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:38:46.310065 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfac4ce6-7c7c-4ce6-8f5e-a692836c9c1d None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:38:46.310843 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfac4ce6-7c7c-4ce6-8f5e-a692836c9c1d None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:38:46.310843 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfac4ce6-7c7c-4ce6-8f5e-a692836c9c1d None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:38:46.312049 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 725/1447] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 22:38:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:38:46.317714 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fc6673-539d-45f5-8ee2-edf926b0c707 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:38:46.318170 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fc6673-539d-45f5-8ee2-edf926b0c707 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:38:46.318273 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fc6673-539d-45f5-8ee2-edf926b0c707 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:38:46.366238 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:38:46.384649 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-01fc6673-539d-45f5-8ee2-edf926b0c707 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:38:46.386521 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-01fc6673-539d-45f5-8ee2-edf926b0c707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:38:46.483687 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 723/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 22:38:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 22:38:46.523569 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfac4ce6-7c7c-4ce6-8f5e-a692836c9c1d None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:38:46.525091 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-dfac4ce6-7c7c-4ce6-8f5e-a692836c9c1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:38:46.627718 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfac4ce6-7c7c-4ce6-8f5e-a692836c9c1d None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:38:46.668312 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:38:46.706747 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5236514f-32dc-4b38-ae3d-2143c44513e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:38:46.708685 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5236514f-32dc-4b38-ae3d-2143c44513e7 service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:38:46.708970 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5236514f-32dc-4b38-ae3d-2143c44513e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:38:46.709246 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5236514f-32dc-4b38-ae3d-2143c44513e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:38:46.711037 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-5236514f-32dc-4b38-ae3d-2143c44513e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:38:46.833561 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5236514f-32dc-4b38-ae3d-2143c44513e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:38:46.835000 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5236514f-32dc-4b38-ae3d-2143c44513e7 service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:38:46.853258 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 726/1449] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:38:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:38:49.035594 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dfabdb-41f0-4f12-a2fc-ce0604c6c099 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:38:49.036170 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dfabdb-41f0-4f12-a2fc-ce0604c6c099 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:38:49.036272 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dfabdb-41f0-4f12-a2fc-ce0604c6c099 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:38:49.037910 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 724/1450] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 22:38:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:38:49.043350 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbcd66f-634f-47c6-8af6-b243a97517ad None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:38:49.043822 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbcd66f-634f-47c6-8af6-b243a97517ad None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:38:49.043935 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbcd66f-634f-47c6-8af6-b243a97517ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:38:49.116623 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-0bbcd66f-634f-47c6-8af6-b243a97517ad None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:38:49.118219 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-0bbcd66f-634f-47c6-8af6-b243a97517ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:38:49.141833 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:38:49.219948 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 727/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 22:38:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:38:49.258979 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dfabdb-41f0-4f12-a2fc-ce0604c6c099 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:38:49.260531 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-54dfabdb-41f0-4f12-a2fc-ce0604c6c099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:38:49.363225 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dfabdb-41f0-4f12-a2fc-ce0604c6c099 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:38:49.382732 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:38:49.438935 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9649d073-ff9c-4303-81d8-cb095d47e502 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:38:49.439629 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9649d073-ff9c-4303-81d8-cb095d47e502 service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:38:49.439915 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9649d073-ff9c-4303-81d8-cb095d47e502 service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:38:49.440192 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9649d073-ff9c-4303-81d8-cb095d47e502 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:38:49.442330 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-9649d073-ff9c-4303-81d8-cb095d47e502 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:38:49.548278 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9649d073-ff9c-4303-81d8-cb095d47e502 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:38:49.549713 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9649d073-ff9c-4303-81d8-cb095d47e502 service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:38:49.567740 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 725/1452] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:38:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:38:51.659896 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52e868a-f73c-4269-9f44-da3b5f4b1592 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:38:51.660485 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52e868a-f73c-4269-9f44-da3b5f4b1592 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:38:51.660485 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52e868a-f73c-4269-9f44-da3b5f4b1592 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:38:51.661866 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 728/1453] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 22:38:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:38:51.667630 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26659ce1-1058-4b82-b084-6d86ad808690 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:38:51.668040 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26659ce1-1058-4b82-b084-6d86ad808690 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:38:51.668207 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26659ce1-1058-4b82-b084-6d86ad808690 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:38:51.729820 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:38:51.732451 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-26659ce1-1058-4b82-b084-6d86ad808690 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:38:51.733657 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-26659ce1-1058-4b82-b084-6d86ad808690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:38:51.830091 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 726/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 22:38:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:38:51.870396 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52e868a-f73c-4269-9f44-da3b5f4b1592 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:38:51.871976 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-a52e868a-f73c-4269-9f44-da3b5f4b1592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:38:51.978168 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52e868a-f73c-4269-9f44-da3b5f4b1592 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:38:52.027141 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:38:52.052426 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88bb27d-b9da-48ce-afad-97a9a55cc998 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:38:52.053076 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88bb27d-b9da-48ce-afad-97a9a55cc998 service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:38:52.053361 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88bb27d-b9da-48ce-afad-97a9a55cc998 service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:38:52.053638 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88bb27d-b9da-48ce-afad-97a9a55cc998 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:38:52.055302 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-b88bb27d-b9da-48ce-afad-97a9a55cc998 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:38:52.163341 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88bb27d-b9da-48ce-afad-97a9a55cc998 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:38:52.164841 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88bb27d-b9da-48ce-afad-97a9a55cc998 service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:38:52.183493 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 729/1455] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:38:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:38:54.312991 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cb0c39-a252-482e-ad58-0c9c5959198a None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:38:54.313531 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cb0c39-a252-482e-ad58-0c9c5959198a None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:38:54.313531 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cb0c39-a252-482e-ad58-0c9c5959198a None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:38:54.315059 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 727/1456] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 22:38:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:38:54.320895 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42b5b70-3f71-4b09-a6e5-eeecfc0b40dc None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:38:54.321342 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42b5b70-3f71-4b09-a6e5-eeecfc0b40dc None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:38:54.321448 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42b5b70-3f71-4b09-a6e5-eeecfc0b40dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:38:54.386723 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-a42b5b70-3f71-4b09-a6e5-eeecfc0b40dc None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:38:54.387980 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-a42b5b70-3f71-4b09-a6e5-eeecfc0b40dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:38:54.416932 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:38:54.484714 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 730/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 22:38:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:38:54.523685 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cb0c39-a252-482e-ad58-0c9c5959198a None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:38:54.525210 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-27cb0c39-a252-482e-ad58-0c9c5959198a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:38:54.626972 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cb0c39-a252-482e-ad58-0c9c5959198a None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:38:54.655299 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:38:54.699668 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a78d358-8fe7-4297-b390-71db99e9e534 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:38:54.700322 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a78d358-8fe7-4297-b390-71db99e9e534 service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:38:54.700636 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a78d358-8fe7-4297-b390-71db99e9e534 service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:38:54.700935 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a78d358-8fe7-4297-b390-71db99e9e534 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:38:54.702549 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-8a78d358-8fe7-4297-b390-71db99e9e534 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:38:54.811371 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a78d358-8fe7-4297-b390-71db99e9e534 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:38:54.812813 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a78d358-8fe7-4297-b390-71db99e9e534 service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:38:54.831438 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 728/1458] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:38:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:38:57.009288 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71031fa-52ca-4ed7-bb3c-8abe67a9a4a5 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:38:57.009863 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71031fa-52ca-4ed7-bb3c-8abe67a9a4a5 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:38:57.009863 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71031fa-52ca-4ed7-bb3c-8abe67a9a4a5 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:38:57.011265 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 731/1459] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 22:38:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:38:57.016517 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ca5dbe-dd31-4ca1-93bc-4e6d7b12af6e None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:38:57.016960 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ca5dbe-dd31-4ca1-93bc-4e6d7b12af6e None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:38:57.017096 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ca5dbe-dd31-4ca1-93bc-4e6d7b12af6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:38:57.081212 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-39ca5dbe-dd31-4ca1-93bc-4e6d7b12af6e None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:38:57.082441 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-39ca5dbe-dd31-4ca1-93bc-4e6d7b12af6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:38:57.092653 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:38:57.180269 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 729/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 22:38:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:38:57.218700 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71031fa-52ca-4ed7-bb3c-8abe67a9a4a5 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:38:57.220278 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-a71031fa-52ca-4ed7-bb3c-8abe67a9a4a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:38:57.326724 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71031fa-52ca-4ed7-bb3c-8abe67a9a4a5 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:38:57.382163 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:38:57.401372 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956bf0e4-1ff7-423b-b633-bd7efdd06ee1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:38:57.402027 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956bf0e4-1ff7-423b-b633-bd7efdd06ee1 service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:38:57.402382 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956bf0e4-1ff7-423b-b633-bd7efdd06ee1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:38:57.402857 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956bf0e4-1ff7-423b-b633-bd7efdd06ee1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:38:57.404954 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-956bf0e4-1ff7-423b-b633-bd7efdd06ee1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:38:57.510778 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956bf0e4-1ff7-423b-b633-bd7efdd06ee1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:38:57.512246 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956bf0e4-1ff7-423b-b633-bd7efdd06ee1 service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:38:57.529731 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 732/1461] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:38:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:38:59.624210 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a19e6af-8353-484b-a2b9-4dc360d6d1f6 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:38:59.624750 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a19e6af-8353-484b-a2b9-4dc360d6d1f6 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:38:59.624875 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a19e6af-8353-484b-a2b9-4dc360d6d1f6 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:38:59.626188 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 730/1462] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 22:38:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:38:59.632235 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7201da7-575f-49c7-9564-b8807a0d2453 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:38:59.632655 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7201da7-575f-49c7-9564-b8807a0d2453 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:38:59.632819 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7201da7-575f-49c7-9564-b8807a0d2453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:38:59.696057 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-d7201da7-575f-49c7-9564-b8807a0d2453 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:38:59.697309 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-d7201da7-575f-49c7-9564-b8807a0d2453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:38:59.734311 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:38:59.792465 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 733/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 22:38:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:38:59.831293 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a19e6af-8353-484b-a2b9-4dc360d6d1f6 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:38:59.832818 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-9a19e6af-8353-484b-a2b9-4dc360d6d1f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:38:59.934326 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a19e6af-8353-484b-a2b9-4dc360d6d1f6 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:38:59.970800 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:39:00.006580 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9e8f57-c8be-478e-989c-bb49960433f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:39:00.007259 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9e8f57-c8be-478e-989c-bb49960433f2 service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:39:00.007543 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9e8f57-c8be-478e-989c-bb49960433f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:39:00.007818 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9e8f57-c8be-478e-989c-bb49960433f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:39:00.009466 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-ef9e8f57-c8be-478e-989c-bb49960433f2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:39:00.113892 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9e8f57-c8be-478e-989c-bb49960433f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:39:00.115839 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9e8f57-c8be-478e-989c-bb49960433f2 service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:39:00.134844 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 731/1464] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:38:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:39:02.228021 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c158f628-292b-4bb5-a8e8-2ab770fcff01 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:39:02.228524 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c158f628-292b-4bb5-a8e8-2ab770fcff01 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:39:02.228636 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c158f628-292b-4bb5-a8e8-2ab770fcff01 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:39:02.229925 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 734/1465] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 22:39:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:39:02.235290 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3d8aed-90d1-4e09-a727-4dc471cb7697 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:39:02.235693 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3d8aed-90d1-4e09-a727-4dc471cb7697 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:39:02.235865 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3d8aed-90d1-4e09-a727-4dc471cb7697 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:39:02.307263 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-ba3d8aed-90d1-4e09-a727-4dc471cb7697 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:39:02.308507 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-ba3d8aed-90d1-4e09-a727-4dc471cb7697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:39:02.326218 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:39:02.404253 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 732/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 22:39:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:39:02.442345 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c158f628-292b-4bb5-a8e8-2ab770fcff01 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:39:02.443948 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-c158f628-292b-4bb5-a8e8-2ab770fcff01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:39:02.547262 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c158f628-292b-4bb5-a8e8-2ab770fcff01 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:39:02.613371 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:39:02.620334 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc19776-7dfb-475c-a68a-8459cc8a91ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:39:02.620984 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc19776-7dfb-475c-a68a-8459cc8a91ef service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:39:02.621329 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc19776-7dfb-475c-a68a-8459cc8a91ef service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:39:02.621612 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc19776-7dfb-475c-a68a-8459cc8a91ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:39:02.623408 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-efc19776-7dfb-475c-a68a-8459cc8a91ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:39:02.732382 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc19776-7dfb-475c-a68a-8459cc8a91ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:39:02.733932 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc19776-7dfb-475c-a68a-8459cc8a91ef service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:39:02.753524 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 735/1467] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:39:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:39:04.865915 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee00e05-f9ac-4b1c-989b-333916c2551b None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:39:04.866581 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee00e05-f9ac-4b1c-989b-333916c2551b None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:39:04.866581 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee00e05-f9ac-4b1c-989b-333916c2551b None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:39:04.867864 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 733/1468] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 22:39:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:39:04.873196 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb087ae8-e62e-4ed8-ad43-1671528c9986 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:39:04.873652 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb087ae8-e62e-4ed8-ad43-1671528c9986 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:39:04.873763 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb087ae8-e62e-4ed8-ad43-1671528c9986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:39:04.937280 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-fb087ae8-e62e-4ed8-ad43-1671528c9986 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:39:04.938495 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-fb087ae8-e62e-4ed8-ad43-1671528c9986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:39:04.987833 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:39:05.037326 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 736/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 22:39:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:39:05.076080 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee00e05-f9ac-4b1c-989b-333916c2551b None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:39:05.077671 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-bee00e05-f9ac-4b1c-989b-333916c2551b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:39:05.181325 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee00e05-f9ac-4b1c-989b-333916c2551b None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:39:05.231288 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:39:05.258093 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad16ae4-8ccf-4ab9-a6cd-df971eaaf2fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:39:05.258752 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad16ae4-8ccf-4ab9-a6cd-df971eaaf2fb service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:39:05.259061 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad16ae4-8ccf-4ab9-a6cd-df971eaaf2fb service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:39:05.259347 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad16ae4-8ccf-4ab9-a6cd-df971eaaf2fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:39:05.260984 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-bad16ae4-8ccf-4ab9-a6cd-df971eaaf2fb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:39:05.369744 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad16ae4-8ccf-4ab9-a6cd-df971eaaf2fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:39:05.371208 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad16ae4-8ccf-4ab9-a6cd-df971eaaf2fb service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:39:05.389239 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 734/1470] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:39:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:39:15.035530 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=22 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:39:15.386848 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=66 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:39:25.237729 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed29302-ed14-4657-921a-deee449167c9 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:39:25.238242 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed29302-ed14-4657-921a-deee449167c9 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:39:25.238357 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed29302-ed14-4657-921a-deee449167c9 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:39:25.239698 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 737/1471] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:39:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:39:25.244983 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1e037d-1aec-400b-a48a-3dfed63b045c None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:39:25.245429 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1e037d-1aec-400b-a48a-3dfed63b045c None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:39:25.245527 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1e037d-1aec-400b-a48a-3dfed63b045c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:39:25.311314 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-1f1e037d-1aec-400b-a48a-3dfed63b045c None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:39:25.312532 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-1f1e037d-1aec-400b-a48a-3dfed63b045c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:39:25.408159 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 735/1472] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:39:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:39:25.493406 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee697c5-2d30-4dbb-b7f2-37f0a0313bdb None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:39:25.493697 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee697c5-2d30-4dbb-b7f2-37f0a0313bdb None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:39:25.493976 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee697c5-2d30-4dbb-b7f2-37f0a0313bdb None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:39:25.495393 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 738/1473] 10.222.0.81 () {54 vars in 966 bytes} [Wed Jul 20 22:39:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:39:25.500269 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3536e75a-9958-48b1-992b-ada7f21ef349 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:39:25.500567 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3536e75a-9958-48b1-992b-ada7f21ef349 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:39:25.500873 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3536e75a-9958-48b1-992b-ada7f21ef349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:39:25.574284 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-3536e75a-9958-48b1-992b-ada7f21ef349 None None] MFA Rules not processed for user `9c44640ea0014426a9523bb1c457778b`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:39:25.575523 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-3536e75a-9958-48b1-992b-ada7f21ef349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:39:25.596082 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:39:25.651608 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 736/1474] 10.222.0.81 () {58 vars in 1063 bytes} [Wed Jul 20 22:39:25 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:39:25.658883 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee697c5-2d30-4dbb-b7f2-37f0a0313bdb None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:39:25.660423 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-dee697c5-2d30-4dbb-b7f2-37f0a0313bdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:39:25.764155 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee697c5-2d30-4dbb-b7f2-37f0a0313bdb None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:39:25.836785 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fda1962-a061-4bf8-b849-4e0c2a8acf85 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c44640ea0014426a9523bb1c457778b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:39:25.837431 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fda1962-a061-4bf8-b849-4e0c2a8acf85 service glance] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:39:25.837717 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fda1962-a061-4bf8-b849-4e0c2a8acf85 service glance] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:39:25.837996 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fda1962-a061-4bf8-b849-4e0c2a8acf85 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:39:25.839659 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-2fda1962-a061-4bf8-b849-4e0c2a8acf85 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:39:25.892897 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:39:25.947642 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fda1962-a061-4bf8-b849-4e0c2a8acf85 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:39:25.949074 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fda1962-a061-4bf8-b849-4e0c2a8acf85 service glance] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:39:25.968474 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 739/1475] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:39:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:39:28.115985 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c93c97-8548-4bfb-a599-eb71ccace908 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:39:28.116575 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c93c97-8548-4bfb-a599-eb71ccace908 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:39:28.116575 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c93c97-8548-4bfb-a599-eb71ccace908 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:39:28.117940 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 737/1476] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:39:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:39:28.123320 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0152e193-62ee-4d8e-a378-87f3e2c086d5 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:39:28.123780 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0152e193-62ee-4d8e-a378-87f3e2c086d5 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:39:28.123894 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0152e193-62ee-4d8e-a378-87f3e2c086d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:39:28.187911 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-0152e193-62ee-4d8e-a378-87f3e2c086d5 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:39:28.189126 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-0152e193-62ee-4d8e-a378-87f3e2c086d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:39:28.292366 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 740/1477] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:39:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:39:28.316697 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c93c97-8548-4bfb-a599-eb71ccace908 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:39:28.318309 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-f7c93c97-8548-4bfb-a599-eb71ccace908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:39:28.445400 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c93c97-8548-4bfb-a599-eb71ccace908 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:39:28.496567 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:39:28.557120 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23befd3-47f0-4bbc-8c5a-c04b79c678c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:39:28.562039 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23befd3-47f0-4bbc-8c5a-c04b79c678c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:39:28.562334 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23befd3-47f0-4bbc-8c5a-c04b79c678c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:39:28.562609 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23befd3-47f0-4bbc-8c5a-c04b79c678c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:39:28.563389 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23befd3-47f0-4bbc-8c5a-c04b79c678c9 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:39:28.565178 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: WARNING py.warnings [None req-a23befd3-47f0-4bbc-8c5a-c04b79c678c9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 22:39:28.565178 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: warnings.warn(msg) Jul 20 22:39:28.565178 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: Jul 20 22:39:28.566326 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23befd3-47f0-4bbc-8c5a-c04b79c678c9 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:39:28.590855 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 738/1478] 10.222.0.81 () {56 vars in 1108 bytes} [Wed Jul 20 22:39:28 2022] GET /identity/v3/projects => generated 2553 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:39:30.670124 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3fff1f-7a0d-4c48-9ddf-609d77d29d6b None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:39:30.670709 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3fff1f-7a0d-4c48-9ddf-609d77d29d6b None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:39:30.670709 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3fff1f-7a0d-4c48-9ddf-609d77d29d6b None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:39:30.672114 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 741/1479] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:39:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:39:30.677257 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ad434b-abd5-499d-8f1d-6afa8791037e None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:39:30.678196 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ad434b-abd5-499d-8f1d-6afa8791037e None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:39:30.678310 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ad434b-abd5-499d-8f1d-6afa8791037e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:39:30.742004 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-c1ad434b-abd5-499d-8f1d-6afa8791037e None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:39:30.743258 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-c1ad434b-abd5-499d-8f1d-6afa8791037e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:39:30.838494 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 739/1480] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:39:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:39:30.882063 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3fff1f-7a0d-4c48-9ddf-609d77d29d6b None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:39:30.883608 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-3e3fff1f-7a0d-4c48-9ddf-609d77d29d6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:39:30.893286 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:39:31.028403 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3fff1f-7a0d-4c48-9ddf-609d77d29d6b None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:39:31.107154 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:39:31.139848 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd391d03-f193-4ca4-a1c8-2aa6c84d6107 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7829324db2574d38b18c749d0559177f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:39:31.140494 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd391d03-f193-4ca4-a1c8-2aa6c84d6107 service nova] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:39:31.140782 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd391d03-f193-4ca4-a1c8-2aa6c84d6107 service nova] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:39:31.141067 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd391d03-f193-4ca4-a1c8-2aa6c84d6107 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:39:31.142711 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-cd391d03-f193-4ca4-a1c8-2aa6c84d6107 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:39:31.233168 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd391d03-f193-4ca4-a1c8-2aa6c84d6107 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:39:31.234843 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd391d03-f193-4ca4-a1c8-2aa6c84d6107 service nova] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:39:31.254081 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 742/1481] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:39:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:39:33.466999 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b14fde5-f4e7-4485-aec6-6f81ea05dafe None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:39:33.467563 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b14fde5-f4e7-4485-aec6-6f81ea05dafe None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:39:33.467563 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b14fde5-f4e7-4485-aec6-6f81ea05dafe None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:39:33.468929 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 740/1482] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:39:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:39:33.474259 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4225e373-55e8-491c-a631-2cfb079dc9d9 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:39:33.474717 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4225e373-55e8-491c-a631-2cfb079dc9d9 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:39:33.474818 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4225e373-55e8-491c-a631-2cfb079dc9d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:39:33.562103 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-4225e373-55e8-491c-a631-2cfb079dc9d9 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:39:33.563358 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-4225e373-55e8-491c-a631-2cfb079dc9d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:39:33.609709 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:39:33.662524 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 743/1483] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:39:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:39:33.700390 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b14fde5-f4e7-4485-aec6-6f81ea05dafe None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:39:33.701958 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-7b14fde5-f4e7-4485-aec6-6f81ea05dafe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:39:33.725180 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:39:33.804729 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b14fde5-f4e7-4485-aec6-6f81ea05dafe None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:39:33.880509 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2e5554-d7d3-44fd-8b99-e02b19c21aa2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:39:33.881825 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2e5554-d7d3-44fd-8b99-e02b19c21aa2 service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:39:33.882030 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2e5554-d7d3-44fd-8b99-e02b19c21aa2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:39:33.882299 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2e5554-d7d3-44fd-8b99-e02b19c21aa2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:39:33.884026 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-2b2e5554-d7d3-44fd-8b99-e02b19c21aa2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:39:33.961062 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:39:33.998325 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2e5554-d7d3-44fd-8b99-e02b19c21aa2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:39:34.000007 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2e5554-d7d3-44fd-8b99-e02b19c21aa2 service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:39:34.018235 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 741/1484] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:39:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:39:36.153904 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96944c50-bf13-4140-9c40-a7d1f3bf8a54 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:39:36.154499 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96944c50-bf13-4140-9c40-a7d1f3bf8a54 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:39:36.154605 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96944c50-bf13-4140-9c40-a7d1f3bf8a54 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:39:36.155970 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 744/1485] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:39:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:39:36.161116 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bd4173-e9ea-49c1-9398-40f7318409c8 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:39:36.161589 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bd4173-e9ea-49c1-9398-40f7318409c8 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:39:36.161692 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bd4173-e9ea-49c1-9398-40f7318409c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:39:36.227550 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-70bd4173-e9ea-49c1-9398-40f7318409c8 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:39:36.228785 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-70bd4173-e9ea-49c1-9398-40f7318409c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:39:36.329632 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 742/1486] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:39:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:39:36.368289 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96944c50-bf13-4140-9c40-a7d1f3bf8a54 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:39:36.369850 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-96944c50-bf13-4140-9c40-a7d1f3bf8a54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:39:36.473746 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96944c50-bf13-4140-9c40-a7d1f3bf8a54 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:39:36.545209 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:39:36.547125 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dc4cb1-7424-4403-b91e-40acdd4f9c10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:39:36.547602 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dc4cb1-7424-4403-b91e-40acdd4f9c10 service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:39:36.547885 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dc4cb1-7424-4403-b91e-40acdd4f9c10 service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:39:36.548164 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dc4cb1-7424-4403-b91e-40acdd4f9c10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:39:36.549818 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-99dc4cb1-7424-4403-b91e-40acdd4f9c10 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:39:36.657346 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dc4cb1-7424-4403-b91e-40acdd4f9c10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:39:36.658774 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dc4cb1-7424-4403-b91e-40acdd4f9c10 service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:39:36.676698 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 745/1487] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:39:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:39:38.900315 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df043847-d818-4a34-bac5-ca7e9159e2b6 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:39:38.900888 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df043847-d818-4a34-bac5-ca7e9159e2b6 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:39:38.900888 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df043847-d818-4a34-bac5-ca7e9159e2b6 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:39:38.902258 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 743/1488] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:39:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:39:38.907498 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea68220-2c1b-4834-8484-09cf15213a35 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:39:38.907911 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea68220-2c1b-4834-8484-09cf15213a35 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:39:38.908065 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea68220-2c1b-4834-8484-09cf15213a35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:39:38.973059 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-3ea68220-2c1b-4834-8484-09cf15213a35 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:39:38.974281 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-3ea68220-2c1b-4834-8484-09cf15213a35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:39:39.027475 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:39:39.071416 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 746/1489] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:39:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 22:39:39.113525 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df043847-d818-4a34-bac5-ca7e9159e2b6 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:39:39.115198 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-df043847-d818-4a34-bac5-ca7e9159e2b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:39:39.150501 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:39:39.218209 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df043847-d818-4a34-bac5-ca7e9159e2b6 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:39:39.293077 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362ab9ba-160a-4175-a930-801243248276 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:39:39.295979 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362ab9ba-160a-4175-a930-801243248276 service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:39:39.295979 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362ab9ba-160a-4175-a930-801243248276 service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:39:39.295979 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362ab9ba-160a-4175-a930-801243248276 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:39:39.295979 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-362ab9ba-160a-4175-a930-801243248276 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:39:39.374894 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:39:39.403474 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362ab9ba-160a-4175-a930-801243248276 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:39:39.405309 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362ab9ba-160a-4175-a930-801243248276 service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:39:39.423564 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 744/1490] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:39:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:39:41.820712 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571074e2-b840-4241-bccc-51d7eb37f114 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:39:41.821302 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571074e2-b840-4241-bccc-51d7eb37f114 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:39:41.821302 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571074e2-b840-4241-bccc-51d7eb37f114 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:39:41.822700 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 747/1491] 10.222.0.81 () {54 vars in 926 bytes} [Wed Jul 20 22:39:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:39:41.828075 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a90c5a-880a-4fd1-bca3-d91623a590d0 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:39:41.828486 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a90c5a-880a-4fd1-bca3-d91623a590d0 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:39:41.828670 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a90c5a-880a-4fd1-bca3-d91623a590d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:39:41.894799 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-86a90c5a-880a-4fd1-bca3-d91623a590d0 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:39:41.896064 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-86a90c5a-880a-4fd1-bca3-d91623a590d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:39:42.116983 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 745/1492] 10.222.0.81 () {58 vars in 1023 bytes} [Wed Jul 20 22:39:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:39:42.157257 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571074e2-b840-4241-bccc-51d7eb37f114 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:39:42.158792 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-571074e2-b840-4241-bccc-51d7eb37f114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:39:42.263495 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571074e2-b840-4241-bccc-51d7eb37f114 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:39:42.347163 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaca42ff-3002-45f9-9cec-4a1c2018134b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:39:42.347819 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaca42ff-3002-45f9-9cec-4a1c2018134b service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:39:42.348104 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaca42ff-3002-45f9-9cec-4a1c2018134b service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:39:42.348379 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaca42ff-3002-45f9-9cec-4a1c2018134b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:39:42.350562 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-aaca42ff-3002-45f9-9cec-4a1c2018134b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:39:42.363541 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:39:42.464251 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaca42ff-3002-45f9-9cec-4a1c2018134b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:39:42.465698 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaca42ff-3002-45f9-9cec-4a1c2018134b service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:39:42.484483 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 748/1493] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:39:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:39:52.114489 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=82 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:39:52.482144 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=50 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:41:27.985368 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfef4451-3b32-4e9d-ae53-7524afbcbc3b None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:41:27.985925 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfef4451-3b32-4e9d-ae53-7524afbcbc3b None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:41:27.985925 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfef4451-3b32-4e9d-ae53-7524afbcbc3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:41:28.059054 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-cfef4451-3b32-4e9d-ae53-7524afbcbc3b None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:41:28.060266 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-cfef4451-3b32-4e9d-ae53-7524afbcbc3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:41:28.156740 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 746/1494] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:41:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:41:28.164824 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485347ee-c1ba-4941-adb6-a85d36bab448 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:41:28.165615 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485347ee-c1ba-4941-adb6-a85d36bab448 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:41:28.165746 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485347ee-c1ba-4941-adb6-a85d36bab448 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:41:28.240482 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-485347ee-c1ba-4941-adb6-a85d36bab448 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:41:28.241642 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-485347ee-c1ba-4941-adb6-a85d36bab448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:41:28.337895 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 749/1495] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:41:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:41:28.342497 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfef4451-3b32-4e9d-ae53-7524afbcbc3b None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:41:28.343718 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-cfef4451-3b32-4e9d-ae53-7524afbcbc3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:41:28.354990 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfef4451-3b32-4e9d-ae53-7524afbcbc3b None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:41:28.415472 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:41:28.447736 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f669477-efdc-46df-aef5-97767fca05da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:41:28.448404 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f669477-efdc-46df-aef5-97767fca05da admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:41:28.448690 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f669477-efdc-46df-aef5-97767fca05da admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:41:28.449606 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f669477-efdc-46df-aef5-97767fca05da admin admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:41:28.450327 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f669477-efdc-46df-aef5-97767fca05da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:41:28.450636 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f669477-efdc-46df-aef5-97767fca05da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:41:28.451671 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: WARNING py.warnings [None req-8f669477-efdc-46df-aef5-97767fca05da admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 20 22:41:28.451671 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: warnings.warn(msg) Jul 20 22:41:28.451671 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: Jul 20 22:41:28.455625 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f669477-efdc-46df-aef5-97767fca05da admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:41:28.463473 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 747/1496] 10.222.0.81 () {58 vars in 1152 bytes} [Wed Jul 20 22:41:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:41:28.471707 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485347ee-c1ba-4941-adb6-a85d36bab448 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:41:28.481579 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485347ee-c1ba-4941-adb6-a85d36bab448 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:41:28.543451 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:41:28.577347 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be621870-e418-42d2-b92b-695162990bf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:41:28.578011 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be621870-e418-42d2-b92b-695162990bf2 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:41:28.578294 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be621870-e418-42d2-b92b-695162990bf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:41:28.578573 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be621870-e418-42d2-b92b-695162990bf2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:41:28.579291 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be621870-e418-42d2-b92b-695162990bf2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:41:28.581011 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: WARNING py.warnings [None req-be621870-e418-42d2-b92b-695162990bf2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 22:41:28.581011 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: warnings.warn(msg) Jul 20 22:41:28.581011 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: Jul 20 22:41:28.582190 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be621870-e418-42d2-b92b-695162990bf2 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:41:28.708388 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 750/1497] 10.222.0.81 () {60 vars in 1152 bytes} [Wed Jul 20 22:41:28 2022] POST /identity/v3/projects => generated 379 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:41:28.712668 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f669477-efdc-46df-aef5-97767fca05da admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:41:28.714157 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-8f669477-efdc-46df-aef5-97767fca05da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:41:28.747605 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f669477-efdc-46df-aef5-97767fca05da admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:41:28.840631 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1785e015-57d2-4efb-a3fd-ba070c776c04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:41:28.841310 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1785e015-57d2-4efb-a3fd-ba070c776c04 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:41:28.841596 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1785e015-57d2-4efb-a3fd-ba070c776c04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:41:28.841881 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1785e015-57d2-4efb-a3fd-ba070c776c04 admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:41:28.842560 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1785e015-57d2-4efb-a3fd-ba070c776c04 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:41:28.843597 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: WARNING py.warnings [None req-1785e015-57d2-4efb-a3fd-ba070c776c04 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 22:41:28.843597 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: warnings.warn(msg) Jul 20 22:41:28.843597 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: Jul 20 22:41:28.844792 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1785e015-57d2-4efb-a3fd-ba070c776c04 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:41:28.970747 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 748/1498] 10.222.0.81 () {60 vars in 1143 bytes} [Wed Jul 20 22:41:28 2022] POST /identity/v3/users => generated 290 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:41:28.974908 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be621870-e418-42d2-b92b-695162990bf2 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:41:28.978089 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-be621870-e418-42d2-b92b-695162990bf2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:41:29.038646 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be621870-e418-42d2-b92b-695162990bf2 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:41:29.122770 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:41:29.132826 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1405c7-3dc5-46bc-a187-1ede12a1d363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:41:29.137467 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1405c7-3dc5-46bc-a187-1ede12a1d363 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:41:29.137467 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1405c7-3dc5-46bc-a187-1ede12a1d363 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:41:29.137467 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1405c7-3dc5-46bc-a187-1ede12a1d363 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:41:29.137467 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1405c7-3dc5-46bc-a187-1ede12a1d363 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:41:29.137467 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: WARNING py.warnings [None req-1c1405c7-3dc5-46bc-a187-1ede12a1d363 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 20 22:41:29.137467 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: warnings.warn(msg) Jul 20 22:41:29.137467 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: Jul 20 22:41:29.138093 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1405c7-3dc5-46bc-a187-1ede12a1d363 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:41:29.149439 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 751/1499] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:41:28 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:41:29.153508 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1785e015-57d2-4efb-a3fd-ba070c776c04 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:41:29.154999 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-1785e015-57d2-4efb-a3fd-ba070c776c04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:41:29.166566 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1785e015-57d2-4efb-a3fd-ba070c776c04 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:41:29.191403 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:41:29.263351 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcea89a-d74c-470b-95a2-4b4c1f11070c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:41:29.264026 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcea89a-d74c-470b-95a2-4b4c1f11070c admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:41:29.264307 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcea89a-d74c-470b-95a2-4b4c1f11070c admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:41:29.264583 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcea89a-d74c-470b-95a2-4b4c1f11070c admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:41:29.265299 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcea89a-d74c-470b-95a2-4b4c1f11070c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:41:29.266304 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: WARNING py.warnings [None req-6fcea89a-d74c-470b-95a2-4b4c1f11070c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 20 22:41:29.266304 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: warnings.warn(msg) Jul 20 22:41:29.266304 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: Jul 20 22:41:29.267207 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcea89a-d74c-470b-95a2-4b4c1f11070c admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:41:29.279297 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 749/1500] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:41:29 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:41:29.283490 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1405c7-3dc5-46bc-a187-1ede12a1d363 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:41:29.285042 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-1c1405c7-3dc5-46bc-a187-1ede12a1d363 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:41:29.318020 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1405c7-3dc5-46bc-a187-1ede12a1d363 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:41:29.409504 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416ccfec-98eb-4eea-b804-cf393000682c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:41:29.410351 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416ccfec-98eb-4eea-b804-cf393000682c admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:41:29.410635 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416ccfec-98eb-4eea-b804-cf393000682c admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:41:29.410912 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416ccfec-98eb-4eea-b804-cf393000682c admin admin] PATH_INFO: `/v3/projects/baffe7f09e234d83a0370efffed5d6d2/users/afecfda5b6fb4889ac7bac4541ac3155/roles/559942c4de1744169f8b5e0404ecde6b` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:41:29.435502 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416ccfec-98eb-4eea-b804-cf393000682c admin admin] RBAC: Authorizing `identity:create_grant(project_id=baffe7f09e234d83a0370efffed5d6d2, user_id=afecfda5b6fb4889ac7bac4541ac3155, role_id=559942c4de1744169f8b5e0404ecde6b)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:41:29.440974 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: WARNING py.warnings [None req-416ccfec-98eb-4eea-b804-cf393000682c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 22:41:29.440974 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: warnings.warn(msg) Jul 20 22:41:29.440974 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: Jul 20 22:41:29.440974 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416ccfec-98eb-4eea-b804-cf393000682c admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:41:29.572249 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 752/1501] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:41:29 2022] PUT /identity/v3/projects/baffe7f09e234d83a0370efffed5d6d2/users/afecfda5b6fb4889ac7bac4541ac3155/roles/559942c4de1744169f8b5e0404ecde6b => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:41:29.584352 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe2a153-b910-4a5d-a803-82db187ac179 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:41:29.584642 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe2a153-b910-4a5d-a803-82db187ac179 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:41:29.584943 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe2a153-b910-4a5d-a803-82db187ac179 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:41:29.638706 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-bbe2a153-b910-4a5d-a803-82db187ac179 None None] MFA Rules not processed for user `afecfda5b6fb4889ac7bac4541ac3155`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:41:29.639919 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-bbe2a153-b910-4a5d-a803-82db187ac179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:41:29.714952 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 750/1502] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:41:29 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:41:29.728165 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9291f704-1082-40e5-9a5b-015ca90e9337 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:41:29.728485 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9291f704-1082-40e5-9a5b-015ca90e9337 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:41:29.728794 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9291f704-1082-40e5-9a5b-015ca90e9337 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:41:29.730652 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 753/1503] 10.222.0.81 () {54 vars in 964 bytes} [Wed Jul 20 22:41:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:41:29.735599 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b834b2-0cab-47af-9484-814d694bb9f8 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:41:29.735886 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b834b2-0cab-47af-9484-814d694bb9f8 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:41:29.736177 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b834b2-0cab-47af-9484-814d694bb9f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:41:29.765498 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:41:29.810286 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-c4b834b2-0cab-47af-9484-814d694bb9f8 None None] MFA Rules not processed for user `ecf4db289fba4e4a806198c62d31f7c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:41:29.811564 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-c4b834b2-0cab-47af-9484-814d694bb9f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:41:29.888192 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 751/1504] 10.222.0.81 () {58 vars in 1061 bytes} [Wed Jul 20 22:41:29 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:41:29.895937 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9291f704-1082-40e5-9a5b-015ca90e9337 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:41:29.897449 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-9291f704-1082-40e5-9a5b-015ca90e9337 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:41:29.947882 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:41:30.001448 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9291f704-1082-40e5-9a5b-015ca90e9337 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:41:30.074843 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42250d73-0581-45c8-9cd9-5b8a8763739a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecf4db289fba4e4a806198c62d31f7c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:41:30.075538 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42250d73-0581-45c8-9cd9-5b8a8763739a service cinder] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:41:30.075956 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42250d73-0581-45c8-9cd9-5b8a8763739a service cinder] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:41:30.076232 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42250d73-0581-45c8-9cd9-5b8a8763739a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:41:30.077895 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-42250d73-0581-45c8-9cd9-5b8a8763739a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:41:30.175991 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42250d73-0581-45c8-9cd9-5b8a8763739a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:41:30.177427 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42250d73-0581-45c8-9cd9-5b8a8763739a service cinder] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:41:30.181012 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:41:30.196654 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 754/1505] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:41:29 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:41:30.238669 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174ac483-3631-4e76-9b46-3959cd09a3ba None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:41:30.239136 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174ac483-3631-4e76-9b46-3959cd09a3ba None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:41:30.239311 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174ac483-3631-4e76-9b46-3959cd09a3ba None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:41:30.240601 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 752/1506] 10.222.0.81 () {52 vars in 829 bytes} [Wed Jul 20 22:41:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:41:30.244178 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42250d73-0581-45c8-9cd9-5b8a8763739a service cinder] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:41:30.245502 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-42250d73-0581-45c8-9cd9-5b8a8763739a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:41:30.344484 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42250d73-0581-45c8-9cd9-5b8a8763739a service cinder] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:41:30.439557 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d0df60-717d-4e0a-ab80-c5596360682e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:41:30.440430 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d0df60-717d-4e0a-ab80-c5596360682e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:41:30.440714 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d0df60-717d-4e0a-ab80-c5596360682e admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:41:30.440992 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d0df60-717d-4e0a-ab80-c5596360682e admin admin] PATH_INFO: `/v3/users/afecfda5b6fb4889ac7bac4541ac3155` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:41:30.453288 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d0df60-717d-4e0a-ab80-c5596360682e admin admin] RBAC: Authorizing `identity:delete_user(user_id=afecfda5b6fb4889ac7bac4541ac3155)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:41:30.454294 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: WARNING py.warnings [None req-21d0df60-717d-4e0a-ab80-c5596360682e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 22:41:30.454294 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: warnings.warn(msg) Jul 20 22:41:30.454294 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: Jul 20 22:41:30.455555 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d0df60-717d-4e0a-ab80-c5596360682e admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:41:30.465071 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:41:30.674329 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-21d0df60-717d-4e0a-ab80-c5596360682e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'afecfda5b6fb4889ac7bac4541ac3155'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:41:30.674813 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-21d0df60-717d-4e0a-ab80-c5596360682e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'afecfda5b6fb4889ac7bac4541ac3155'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:41:30.675254 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-21d0df60-717d-4e0a-ab80-c5596360682e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'afecfda5b6fb4889ac7bac4541ac3155'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:41:30.686806 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-21d0df60-717d-4e0a-ab80-c5596360682e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'afecfda5b6fb4889ac7bac4541ac3155'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:41:30.753069 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-21d0df60-717d-4e0a-ab80-c5596360682e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'afecfda5b6fb4889ac7bac4541ac3155'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:41:30.776372 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 755/1507] 10.222.0.81 () {58 vars in 1223 bytes} [Wed Jul 20 22:41:30 2022] DELETE /identity/v3/users/afecfda5b6fb4889ac7bac4541ac3155 => generated 0 bytes in 533 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:41:30.781379 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5928da6-310e-4da5-a21d-19781c44a19e None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:41:30.781671 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5928da6-310e-4da5-a21d-19781c44a19e None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:41:30.781954 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5928da6-310e-4da5-a21d-19781c44a19e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:41:30.836620 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-f5928da6-310e-4da5-a21d-19781c44a19e None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:41:30.837856 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-f5928da6-310e-4da5-a21d-19781c44a19e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:41:30.935112 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:41:30.947692 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 753/1508] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:41:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:41:30.957786 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d0df60-717d-4e0a-ab80-c5596360682e admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:41:30.959025 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-21d0df60-717d-4e0a-ab80-c5596360682e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:41:31.059426 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d0df60-717d-4e0a-ab80-c5596360682e admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:41:31.112626 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:41:31.131354 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207ae073-88d3-42c9-8623-db25bc0d3cf5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:41:31.132006 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207ae073-88d3-42c9-8623-db25bc0d3cf5 service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:41:31.132290 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207ae073-88d3-42c9-8623-db25bc0d3cf5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:41:31.132571 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207ae073-88d3-42c9-8623-db25bc0d3cf5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:41:31.134203 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-207ae073-88d3-42c9-8623-db25bc0d3cf5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:41:31.263293 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207ae073-88d3-42c9-8623-db25bc0d3cf5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:41:31.264786 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207ae073-88d3-42c9-8623-db25bc0d3cf5 service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:41:31.285549 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 756/1509] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:41:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:41:31.795776 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5928da6-310e-4da5-a21d-19781c44a19e None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:41:31.797049 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-f5928da6-310e-4da5-a21d-19781c44a19e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:41:31.810415 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5928da6-310e-4da5-a21d-19781c44a19e None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:41:31.922495 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2447d444-ff92-4c8f-8c2a-89847d556240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:41:31.923418 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2447d444-ff92-4c8f-8c2a-89847d556240 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:41:31.923696 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2447d444-ff92-4c8f-8c2a-89847d556240 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:41:31.924131 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2447d444-ff92-4c8f-8c2a-89847d556240 admin admin] PATH_INFO: `/v3/projects/baffe7f09e234d83a0370efffed5d6d2` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:41:31.934747 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2447d444-ff92-4c8f-8c2a-89847d556240 admin admin] RBAC: Authorizing `identity:delete_project(project_id=baffe7f09e234d83a0370efffed5d6d2)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:41:31.935892 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: WARNING py.warnings [None req-2447d444-ff92-4c8f-8c2a-89847d556240 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 22:41:31.935892 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: warnings.warn(msg) Jul 20 22:41:31.935892 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: Jul 20 22:41:31.937079 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2447d444-ff92-4c8f-8c2a-89847d556240 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:41:32.089834 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-2447d444-ff92-4c8f-8c2a-89847d556240 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'baffe7f09e234d83a0370efffed5d6d2', 'request_id': 'req-2447d444-ff92-4c8f-8c2a-89847d556240'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:41:32.103875 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-2447d444-ff92-4c8f-8c2a-89847d556240 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'baffe7f09e234d83a0370efffed5d6d2', 'request_id': 'req-2447d444-ff92-4c8f-8c2a-89847d556240'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:41:32.216462 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-2447d444-ff92-4c8f-8c2a-89847d556240 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'baffe7f09e234d83a0370efffed5d6d2', 'request_id': 'req-2447d444-ff92-4c8f-8c2a-89847d556240'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:41:32.241971 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-2447d444-ff92-4c8f-8c2a-89847d556240 admin admin] The token cache is being invalidate because project baffe7f09e234d83a0370efffed5d6d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:41:32.242264 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-2447d444-ff92-4c8f-8c2a-89847d556240 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:41:32.242801 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-2447d444-ff92-4c8f-8c2a-89847d556240 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:41:32.244363 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 754/1510] 10.222.0.81 () {58 vars in 1232 bytes} [Wed Jul 20 22:41:31 2022] DELETE /identity/v3/projects/baffe7f09e234d83a0370efffed5d6d2 => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:41:41.283137 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=62 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:41:42.239718 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:43:17.721054 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037467a9-267c-4f44-9f70-d17dc7f06fee None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:43:17.721669 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037467a9-267c-4f44-9f70-d17dc7f06fee None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:43:17.721669 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037467a9-267c-4f44-9f70-d17dc7f06fee None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:43:17.723057 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 757/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 22:43:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:43:17.728050 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cfc83d-8ca7-4f9e-8f5e-c6a81938ccb7 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:43:17.728498 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cfc83d-8ca7-4f9e-8f5e-c6a81938ccb7 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:43:17.728603 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cfc83d-8ca7-4f9e-8f5e-c6a81938ccb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:43:17.775685 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bd9a86-373d-459b-bf5d-932994ec1e21 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:43:17.776063 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bd9a86-373d-459b-bf5d-932994ec1e21 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:43:17.776407 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bd9a86-373d-459b-bf5d-932994ec1e21 None None] PATH_INFO: `/v3` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:43:17.777777 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 758/1512] 10.222.0.23 () {54 vars in 925 bytes} [Wed Jul 20 22:43:17 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 22:43:17.786633 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9060e63-e3b8-4ad5-be99-c9e111549ef5 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:43:17.786921 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9060e63-e3b8-4ad5-be99-c9e111549ef5 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:43:17.787228 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9060e63-e3b8-4ad5-be99-c9e111549ef5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:43:17.813206 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-62cfc83d-8ca7-4f9e-8f5e-c6a81938ccb7 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:43:17.814397 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-62cfc83d-8ca7-4f9e-8f5e-c6a81938ccb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:43:17.849255 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-a9060e63-e3b8-4ad5-be99-c9e111549ef5 None None] MFA Rules not processed for user `d5500cebdd9d4f8e9aecb45d6c955878`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:43:17.850651 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-a9060e63-e3b8-4ad5-be99-c9e111549ef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:43:17.919903 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 755/1513] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 22:43:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:43:17.942631 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 759/1514] 10.222.0.23 () {58 vars in 1015 bytes} [Wed Jul 20 22:43:17 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 22:43:18.074513 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cfc83d-8ca7-4f9e-8f5e-c6a81938ccb7 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:43:18.075913 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-62cfc83d-8ca7-4f9e-8f5e-c6a81938ccb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:43:18.143879 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:43:18.179417 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cfc83d-8ca7-4f9e-8f5e-c6a81938ccb7 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:43:18.185151 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2fd80f-78c5-41ef-bc98-8b677c13d0e4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7829324db2574d38b18c749d0559177f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:43:18.185877 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2fd80f-78c5-41ef-bc98-8b677c13d0e4 service nova] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:43:18.186170 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2fd80f-78c5-41ef-bc98-8b677c13d0e4 service nova] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:43:18.186446 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2fd80f-78c5-41ef-bc98-8b677c13d0e4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:43:18.195444 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2fd80f-78c5-41ef-bc98-8b677c13d0e4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:43:18.196782 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2fd80f-78c5-41ef-bc98-8b677c13d0e4 service nova] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:43:18.206689 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 756/1515] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:43:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:43:19.040509 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a30855-83bf-4c1b-b4ef-4a2f21b5f1f4 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:43:19.041058 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a30855-83bf-4c1b-b4ef-4a2f21b5f1f4 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:43:19.041058 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a30855-83bf-4c1b-b4ef-4a2f21b5f1f4 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:43:19.042634 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 760/1516] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 22:43:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:43:19.048776 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615ac7ee-3c7a-485f-8be1-b4033ef56518 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:43:19.049226 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615ac7ee-3c7a-485f-8be1-b4033ef56518 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:43:19.049325 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615ac7ee-3c7a-485f-8be1-b4033ef56518 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:43:19.067215 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-615ac7ee-3c7a-485f-8be1-b4033ef56518 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:43:19.068420 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-615ac7ee-3c7a-485f-8be1-b4033ef56518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:43:19.076262 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 757/1517] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 22:43:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:43:19.226172 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a30855-83bf-4c1b-b4ef-4a2f21b5f1f4 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:43:19.241591 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a30855-83bf-4c1b-b4ef-4a2f21b5f1f4 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:43:19.247188 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac98329-fd6a-40d2-b27b-5bac6be26ea7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7829324db2574d38b18c749d0559177f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:43:19.248664 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac98329-fd6a-40d2-b27b-5bac6be26ea7 service nova] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:43:19.248977 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac98329-fd6a-40d2-b27b-5bac6be26ea7 service nova] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:43:19.249293 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac98329-fd6a-40d2-b27b-5bac6be26ea7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:43:19.258177 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac98329-fd6a-40d2-b27b-5bac6be26ea7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:43:19.259863 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac98329-fd6a-40d2-b27b-5bac6be26ea7 service nova] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:43:19.269277 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 761/1518] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:43:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:43:27.496337 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ed0e48-b285-4fdc-b04d-e10eca18707b None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:43:27.496864 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ed0e48-b285-4fdc-b04d-e10eca18707b None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:43:27.496864 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ed0e48-b285-4fdc-b04d-e10eca18707b None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:43:27.498186 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 758/1519] 10.222.0.81 () {54 vars in 961 bytes} [Wed Jul 20 22:43:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:43:27.503303 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ab9ba5-c74a-4688-95fd-a2bf240b22b5 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:43:27.503746 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ab9ba5-c74a-4688-95fd-a2bf240b22b5 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:43:27.503856 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ab9ba5-c74a-4688-95fd-a2bf240b22b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:43:27.521614 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-54ab9ba5-c74a-4688-95fd-a2bf240b22b5 None None] MFA Rules not processed for user `d5500cebdd9d4f8e9aecb45d6c955878`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:43:27.522806 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-54ab9ba5-c74a-4688-95fd-a2bf240b22b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:43:27.531412 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 762/1520] 10.222.0.81 () {58 vars in 1058 bytes} [Wed Jul 20 22:43:27 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:43:27.539386 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ed0e48-b285-4fdc-b04d-e10eca18707b None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:43:27.557309 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ed0e48-b285-4fdc-b04d-e10eca18707b None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:43:27.562350 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179d73d8-b652-484a-b87c-c02b5668e729 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5500cebdd9d4f8e9aecb45d6c955878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:43:27.562985 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179d73d8-b652-484a-b87c-c02b5668e729 service placement] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:43:27.563309 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179d73d8-b652-484a-b87c-c02b5668e729 service placement] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:43:27.563585 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179d73d8-b652-484a-b87c-c02b5668e729 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:43:27.571863 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179d73d8-b652-484a-b87c-c02b5668e729 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:43:27.573443 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179d73d8-b652-484a-b87c-c02b5668e729 service placement] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:43:27.582939 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 759/1521] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:43:27 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 22:43:30.496924 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc057ac-37ae-44e1-a73b-1773109c945a None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:43:30.497463 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc057ac-37ae-44e1-a73b-1773109c945a None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:43:30.497463 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc057ac-37ae-44e1-a73b-1773109c945a None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:43:30.498797 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 763/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 22:43:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:43:30.504144 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e25e89-5cb1-431b-a90a-ce05f5b6c66f None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:43:30.504539 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e25e89-5cb1-431b-a90a-ce05f5b6c66f None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:43:30.504711 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e25e89-5cb1-431b-a90a-ce05f5b6c66f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:43:30.522856 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-22e25e89-5cb1-431b-a90a-ce05f5b6c66f None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:43:30.524095 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-22e25e89-5cb1-431b-a90a-ce05f5b6c66f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:43:30.531922 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 760/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 22:43:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 22:43:30.683596 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc057ac-37ae-44e1-a73b-1773109c945a None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:43:30.698245 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc057ac-37ae-44e1-a73b-1773109c945a None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:43:30.703628 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b728d970-cfa9-4917-a5ea-63a1dcbe45f0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7829324db2574d38b18c749d0559177f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:43:30.704285 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b728d970-cfa9-4917-a5ea-63a1dcbe45f0 service nova] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:43:30.704559 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b728d970-cfa9-4917-a5ea-63a1dcbe45f0 service nova] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:43:30.704837 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b728d970-cfa9-4917-a5ea-63a1dcbe45f0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:43:30.713541 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b728d970-cfa9-4917-a5ea-63a1dcbe45f0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:43:30.715168 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b728d970-cfa9-4917-a5ea-63a1dcbe45f0 service nova] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:43:30.724476 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 764/1524] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:43:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:43:31.551836 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a026263-9a39-40e3-a4d6-997237556f55 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:43:31.552364 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a026263-9a39-40e3-a4d6-997237556f55 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:43:31.552364 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a026263-9a39-40e3-a4d6-997237556f55 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:43:31.553705 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 761/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 22:43:31 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:43:31.558734 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1382bb3e-5f47-4e5d-acf2-e3bb89b8d621 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:43:31.559270 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1382bb3e-5f47-4e5d-acf2-e3bb89b8d621 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:43:31.559406 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1382bb3e-5f47-4e5d-acf2-e3bb89b8d621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:43:31.577157 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-1382bb3e-5f47-4e5d-acf2-e3bb89b8d621 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:43:31.578362 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-1382bb3e-5f47-4e5d-acf2-e3bb89b8d621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:43:31.586786 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 765/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 22:43:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:43:31.738377 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a026263-9a39-40e3-a4d6-997237556f55 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:43:31.752691 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a026263-9a39-40e3-a4d6-997237556f55 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:43:31.758125 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dcd4ee-fead-4462-839c-41a19b1ed763 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7829324db2574d38b18c749d0559177f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:43:31.758763 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dcd4ee-fead-4462-839c-41a19b1ed763 service nova] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:43:31.759075 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dcd4ee-fead-4462-839c-41a19b1ed763 service nova] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:43:31.759358 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dcd4ee-fead-4462-839c-41a19b1ed763 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:43:31.767457 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dcd4ee-fead-4462-839c-41a19b1ed763 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:43:31.769561 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36dcd4ee-fead-4462-839c-41a19b1ed763 service nova] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:43:31.778848 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 762/1527] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:43:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:43:36.047549 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4f5f9f-f60e-4004-a1c3-d3ceb6baf583 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:43:36.048119 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4f5f9f-f60e-4004-a1c3-d3ceb6baf583 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:43:36.048119 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4f5f9f-f60e-4004-a1c3-d3ceb6baf583 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:43:36.051664 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 766/1528] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 22:43:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:43:36.058209 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67dea15-f51f-435b-babe-b386696e9782 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:43:36.058680 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67dea15-f51f-435b-babe-b386696e9782 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:43:36.058788 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67dea15-f51f-435b-babe-b386696e9782 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:43:36.076575 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-c67dea15-f51f-435b-babe-b386696e9782 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:43:36.077785 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-c67dea15-f51f-435b-babe-b386696e9782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:43:36.086364 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 763/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 22:43:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:43:36.158382 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4f5f9f-f60e-4004-a1c3-d3ceb6baf583 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:43:36.160687 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-0d4f5f9f-f60e-4004-a1c3-d3ceb6baf583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:43:36.225676 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:43:36.238421 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4f5f9f-f60e-4004-a1c3-d3ceb6baf583 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:43:36.241994 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f9dfde-024e-4f24-b3ab-7a3bb4136c98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:43:36.243471 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f9dfde-024e-4f24-b3ab-7a3bb4136c98 service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:43:36.243471 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f9dfde-024e-4f24-b3ab-7a3bb4136c98 service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:43:36.243471 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f9dfde-024e-4f24-b3ab-7a3bb4136c98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:43:36.251989 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f9dfde-024e-4f24-b3ab-7a3bb4136c98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:43:36.253494 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f9dfde-024e-4f24-b3ab-7a3bb4136c98 service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:43:36.262691 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 767/1530] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:43:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:43:38.290472 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2d9f0c-f471-4e67-b63f-d7f50d391974 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:43:38.291082 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2d9f0c-f471-4e67-b63f-d7f50d391974 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:43:38.291082 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2d9f0c-f471-4e67-b63f-d7f50d391974 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:43:38.292457 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 764/1531] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 22:43:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:43:38.297743 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1222be37-59bd-4ebc-b84e-e45d62e9653a None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:43:38.298189 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1222be37-59bd-4ebc-b84e-e45d62e9653a None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:43:38.298294 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1222be37-59bd-4ebc-b84e-e45d62e9653a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:43:38.316088 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-1222be37-59bd-4ebc-b84e-e45d62e9653a None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:43:38.317286 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-1222be37-59bd-4ebc-b84e-e45d62e9653a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:43:38.325124 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 768/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 22:43:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:43:38.362555 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2d9f0c-f471-4e67-b63f-d7f50d391974 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:43:38.373254 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2d9f0c-f471-4e67-b63f-d7f50d391974 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:43:38.376773 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643975cb-db76-42a4-bed2-2706c6276c86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:43:38.377403 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643975cb-db76-42a4-bed2-2706c6276c86 service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:43:38.377685 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643975cb-db76-42a4-bed2-2706c6276c86 service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:43:38.377957 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643975cb-db76-42a4-bed2-2706c6276c86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:43:38.386846 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643975cb-db76-42a4-bed2-2706c6276c86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:43:38.388282 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643975cb-db76-42a4-bed2-2706c6276c86 service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:43:38.398332 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 765/1533] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:43:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:43:42.396419 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c18a7f-d66d-45f8-8db0-3c641244a88b None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:43:42.397017 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c18a7f-d66d-45f8-8db0-3c641244a88b None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:43:42.397017 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c18a7f-d66d-45f8-8db0-3c641244a88b None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:43:42.398405 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 769/1534] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 22:43:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:43:42.404329 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5664d403-f08d-4630-9613-112a5a1e3486 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:43:42.404897 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5664d403-f08d-4630-9613-112a5a1e3486 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:43:42.404897 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5664d403-f08d-4630-9613-112a5a1e3486 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:43:42.423235 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-5664d403-f08d-4630-9613-112a5a1e3486 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:43:42.424644 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-5664d403-f08d-4630-9613-112a5a1e3486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:43:42.434989 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 766/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 22:43:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:43:42.740493 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c18a7f-d66d-45f8-8db0-3c641244a88b None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:43:42.742217 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-a6c18a7f-d66d-45f8-8db0-3c641244a88b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:43:42.821941 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c18a7f-d66d-45f8-8db0-3c641244a88b None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:43:42.825714 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c63ac8d-34d5-4a2a-98c4-b32e1ff84f66 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecf4db289fba4e4a806198c62d31f7c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:43:42.826398 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c63ac8d-34d5-4a2a-98c4-b32e1ff84f66 service cinder] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:43:42.826703 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c63ac8d-34d5-4a2a-98c4-b32e1ff84f66 service cinder] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:43:42.826983 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c63ac8d-34d5-4a2a-98c4-b32e1ff84f66 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:43:42.835989 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c63ac8d-34d5-4a2a-98c4-b32e1ff84f66 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:43:42.837414 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c63ac8d-34d5-4a2a-98c4-b32e1ff84f66 service cinder] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:43:42.846868 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 770/1536] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:43:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:43:44.896301 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6582cf10-4920-408b-83a5-522de8fc5463 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:43:44.896876 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6582cf10-4920-408b-83a5-522de8fc5463 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:43:44.896876 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6582cf10-4920-408b-83a5-522de8fc5463 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:43:44.898207 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 767/1537] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 22:43:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:43:44.903397 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a15ed1-2795-4f15-9781-b754b87ed25a None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:43:44.903993 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a15ed1-2795-4f15-9781-b754b87ed25a None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:43:44.904227 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a15ed1-2795-4f15-9781-b754b87ed25a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:43:44.922877 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-85a15ed1-2795-4f15-9781-b754b87ed25a None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:43:44.924741 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-85a15ed1-2795-4f15-9781-b754b87ed25a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:43:44.932633 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 771/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 22:43:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:43:45.236878 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6582cf10-4920-408b-83a5-522de8fc5463 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:43:45.238525 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-6582cf10-4920-408b-83a5-522de8fc5463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:43:45.251393 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6582cf10-4920-408b-83a5-522de8fc5463 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:43:45.255596 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f8f0b8-c2b8-4fd8-802b-35bee63dd8ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecf4db289fba4e4a806198c62d31f7c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:43:45.256228 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f8f0b8-c2b8-4fd8-802b-35bee63dd8ed service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:43:45.256511 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f8f0b8-c2b8-4fd8-802b-35bee63dd8ed service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:43:45.256812 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f8f0b8-c2b8-4fd8-802b-35bee63dd8ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:43:45.265665 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f8f0b8-c2b8-4fd8-802b-35bee63dd8ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:43:45.267093 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f8f0b8-c2b8-4fd8-802b-35bee63dd8ed service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:43:45.276488 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 768/1539] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:43:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:43:54.917904 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=49 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:43:55.272268 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=51 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:45:17.199881 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e48884-48d1-47eb-b67f-f86c0118d991 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:17.200908 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e48884-48d1-47eb-b67f-f86c0118d991 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:17.201425 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e48884-48d1-47eb-b67f-f86c0118d991 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:17.212337 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be044bf-cd34-49b2-af02-d5290e99c414 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:17.213326 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be044bf-cd34-49b2-af02-d5290e99c414 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:17.213827 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be044bf-cd34-49b2-af02-d5290e99c414 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:17.227444 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-98e48884-48d1-47eb-b67f-f86c0118d991 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:45:17.228807 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-98e48884-48d1-47eb-b67f-f86c0118d991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:17.241725 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-8be044bf-cd34-49b2-af02-d5290e99c414 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:45:17.242943 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 772/1540] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:45:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:45:17.244744 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-8be044bf-cd34-49b2-af02-d5290e99c414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:17.253240 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32139d1-ab5f-4d04-a004-5faf14781377 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:17.253598 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32139d1-ab5f-4d04-a004-5faf14781377 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:17.253891 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32139d1-ab5f-4d04-a004-5faf14781377 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:17.270043 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 769/1541] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:45:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 22:45:17.288403 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-e32139d1-ab5f-4d04-a004-5faf14781377 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:45:17.289833 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-e32139d1-ab5f-4d04-a004-5faf14781377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:17.293021 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3114fa-9482-480d-8025-138c77c29f12 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:17.293702 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3114fa-9482-480d-8025-138c77c29f12 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:17.294302 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3114fa-9482-480d-8025-138c77c29f12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:17.310460 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 773/1542] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:45:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 22:45:17.318288 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32139d1-ab5f-4d04-a004-5faf14781377 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:17.328479 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32139d1-ab5f-4d04-a004-5faf14781377 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:17.329301 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-1a3114fa-9482-480d-8025-138c77c29f12 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:45:17.333188 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-1a3114fa-9482-480d-8025-138c77c29f12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:17.340513 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09db05c-9cc5-4b3e-ad04-63dc8ec55f30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:17.341193 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09db05c-9cc5-4b3e-ad04-63dc8ec55f30 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:17.341478 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09db05c-9cc5-4b3e-ad04-63dc8ec55f30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:17.341758 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09db05c-9cc5-4b3e-ad04-63dc8ec55f30 admin admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:17.342493 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09db05c-9cc5-4b3e-ad04-63dc8ec55f30 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:45:17.342802 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09db05c-9cc5-4b3e-ad04-63dc8ec55f30 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:17.343863 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: WARNING py.warnings [None req-d09db05c-9cc5-4b3e-ad04-63dc8ec55f30 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 20 22:45:17.343863 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: warnings.warn(msg) Jul 20 22:45:17.343863 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: Jul 20 22:45:17.344752 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09db05c-9cc5-4b3e-ad04-63dc8ec55f30 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:17.348827 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 770/1543] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:45:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:45:17.349966 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136f859e-5990-4661-af1a-27fea7250341 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:17.350261 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136f859e-5990-4661-af1a-27fea7250341 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:17.350541 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136f859e-5990-4661-af1a-27fea7250341 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:17.401295 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 774/1544] 10.222.0.81 () {58 vars in 1152 bytes} [Wed Jul 20 22:45:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:45:17.402156 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09db05c-9cc5-4b3e-ad04-63dc8ec55f30 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:17.407948 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-136f859e-5990-4661-af1a-27fea7250341 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:45:17.410392 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-136f859e-5990-4661-af1a-27fea7250341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:17.421167 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09db05c-9cc5-4b3e-ad04-63dc8ec55f30 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:17.429836 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12042839-6e0e-403e-aff8-a8f04299fb94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:17.432898 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12042839-6e0e-403e-aff8-a8f04299fb94 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:17.432898 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12042839-6e0e-403e-aff8-a8f04299fb94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:17.432898 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12042839-6e0e-403e-aff8-a8f04299fb94 admin admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:17.432898 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12042839-6e0e-403e-aff8-a8f04299fb94 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:45:17.432898 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12042839-6e0e-403e-aff8-a8f04299fb94 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:17.434458 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12042839-6e0e-403e-aff8-a8f04299fb94 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:17.438364 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 771/1545] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:45:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:45:17.439835 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136f859e-5990-4661-af1a-27fea7250341 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:17.458173 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136f859e-5990-4661-af1a-27fea7250341 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:17.467272 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 775/1546] 10.222.0.81 () {58 vars in 1152 bytes} [Wed Jul 20 22:45:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:45:17.467272 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eb790d-a1cd-4a76-b70d-0f6e6ca8318b None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:17.467272 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eb790d-a1cd-4a76-b70d-0f6e6ca8318b None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:17.467272 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eb790d-a1cd-4a76-b70d-0f6e6ca8318b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:17.469995 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5432d6-50e8-47ae-8e50-3cb84ac3d3d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:17.470711 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5432d6-50e8-47ae-8e50-3cb84ac3d3d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:17.471002 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5432d6-50e8-47ae-8e50-3cb84ac3d3d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:17.471300 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5432d6-50e8-47ae-8e50-3cb84ac3d3d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:17.471997 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5432d6-50e8-47ae-8e50-3cb84ac3d3d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:17.473018 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: WARNING py.warnings [None req-bf5432d6-50e8-47ae-8e50-3cb84ac3d3d1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 22:45:17.473018 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: warnings.warn(msg) Jul 20 22:45:17.473018 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: Jul 20 22:45:17.474196 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5432d6-50e8-47ae-8e50-3cb84ac3d3d1 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:17.506910 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-e1eb790d-a1cd-4a76-b70d-0f6e6ca8318b None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:45:17.508171 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-e1eb790d-a1cd-4a76-b70d-0f6e6ca8318b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:17.525675 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 776/1547] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:45:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:45:17.526971 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397511ca-d0b5-40bb-9e01-4c535b7c6927 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:17.527289 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397511ca-d0b5-40bb-9e01-4c535b7c6927 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:17.527569 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397511ca-d0b5-40bb-9e01-4c535b7c6927 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:17.545805 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-397511ca-d0b5-40bb-9e01-4c535b7c6927 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:45:17.548043 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-397511ca-d0b5-40bb-9e01-4c535b7c6927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:17.559573 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 777/1548] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:45:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:45:17.560379 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397511ca-d0b5-40bb-9e01-4c535b7c6927 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:17.571427 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397511ca-d0b5-40bb-9e01-4c535b7c6927 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:17.576087 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187f0649-b4b5-48ae-9633-b5d666a94ff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:17.576740 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187f0649-b4b5-48ae-9633-b5d666a94ff0 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:17.577026 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187f0649-b4b5-48ae-9633-b5d666a94ff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:17.577305 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187f0649-b4b5-48ae-9633-b5d666a94ff0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:17.578049 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187f0649-b4b5-48ae-9633-b5d666a94ff0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:17.580039 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187f0649-b4b5-48ae-9633-b5d666a94ff0 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:17.669158 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 772/1549] 10.222.0.81 () {60 vars in 1152 bytes} [Wed Jul 20 22:45:17 2022] POST /identity/v3/projects => generated 405 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:45:17.670450 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c33cedc-bb4f-4e01-847a-ed70a224a444 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:17.670739 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c33cedc-bb4f-4e01-847a-ed70a224a444 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:17.671030 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c33cedc-bb4f-4e01-847a-ed70a224a444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:17.688096 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-0c33cedc-bb4f-4e01-847a-ed70a224a444 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:45:17.689893 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-0c33cedc-bb4f-4e01-847a-ed70a224a444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:17.742660 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 778/1550] 10.222.0.81 () {60 vars in 1152 bytes} [Wed Jul 20 22:45:17 2022] POST /identity/v3/projects => generated 375 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:45:17.744196 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafa9e3d-8678-4e24-a01f-beaf33052e41 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:17.744493 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafa9e3d-8678-4e24-a01f-beaf33052e41 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:17.744917 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafa9e3d-8678-4e24-a01f-beaf33052e41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:17.750010 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 773/1551] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:45:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:45:17.758527 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c33cedc-bb4f-4e01-847a-ed70a224a444 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:17.763904 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-cafa9e3d-8678-4e24-a01f-beaf33052e41 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:45:17.765344 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-cafa9e3d-8678-4e24-a01f-beaf33052e41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:17.768480 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c33cedc-bb4f-4e01-847a-ed70a224a444 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:17.825784 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d38e76-1cc6-4f6c-a135-aa897e42f578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:17.826463 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d38e76-1cc6-4f6c-a135-aa897e42f578 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:17.826753 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d38e76-1cc6-4f6c-a135-aa897e42f578 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:17.827211 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d38e76-1cc6-4f6c-a135-aa897e42f578 admin admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:17.827947 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d38e76-1cc6-4f6c-a135-aa897e42f578 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:45:17.828262 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d38e76-1cc6-4f6c-a135-aa897e42f578 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:17.829823 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 779/1552] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:45:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:45:17.829979 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d38e76-1cc6-4f6c-a135-aa897e42f578 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:17.830914 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13d9398-7989-443f-af41-2c79a4fe8edf None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:17.831237 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13d9398-7989-443f-af41-2c79a4fe8edf None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:17.831536 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13d9398-7989-443f-af41-2c79a4fe8edf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:17.849793 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 774/1553] 10.222.0.81 () {58 vars in 1152 bytes} [Wed Jul 20 22:45:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:45:17.849793 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5463ef0-84ae-4225-8d35-8b89582ed36a None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:17.849793 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5463ef0-84ae-4225-8d35-8b89582ed36a None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:17.849793 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5463ef0-84ae-4225-8d35-8b89582ed36a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:17.850936 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-f13d9398-7989-443f-af41-2c79a4fe8edf None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:45:17.852273 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-f13d9398-7989-443f-af41-2c79a4fe8edf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:17.860512 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 780/1554] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:45:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:45:17.862277 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd13896-48dc-4af6-b76b-b7a48afe4786 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:17.862570 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd13896-48dc-4af6-b76b-b7a48afe4786 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:17.862856 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd13896-48dc-4af6-b76b-b7a48afe4786 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:17.864751 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-e5463ef0-84ae-4225-8d35-8b89582ed36a None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:45:17.865970 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-e5463ef0-84ae-4225-8d35-8b89582ed36a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:17.876171 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 775/1555] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:45:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:45:17.878161 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5463ef0-84ae-4225-8d35-8b89582ed36a None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:17.881693 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-2bd13896-48dc-4af6-b76b-b7a48afe4786 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:45:17.882922 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-2bd13896-48dc-4af6-b76b-b7a48afe4786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:17.887765 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5463ef0-84ae-4225-8d35-8b89582ed36a None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:17.895118 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 781/1556] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:45:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:45:17.895118 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd13896-48dc-4af6-b76b-b7a48afe4786 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:17.897291 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae641d34-7d53-426c-974a-e96ef4d4a8bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:17.898222 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae641d34-7d53-426c-974a-e96ef4d4a8bb admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:17.898360 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae641d34-7d53-426c-974a-e96ef4d4a8bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:17.898541 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae641d34-7d53-426c-974a-e96ef4d4a8bb admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:17.899278 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae641d34-7d53-426c-974a-e96ef4d4a8bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:17.902099 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae641d34-7d53-426c-974a-e96ef4d4a8bb admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:17.903381 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd13896-48dc-4af6-b76b-b7a48afe4786 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:17.911354 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a06eef-e2a9-4851-a03d-f2af6ab43b88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:17.911354 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a06eef-e2a9-4851-a03d-f2af6ab43b88 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:17.911354 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a06eef-e2a9-4851-a03d-f2af6ab43b88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:17.911354 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a06eef-e2a9-4851-a03d-f2af6ab43b88 admin admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:17.911722 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a06eef-e2a9-4851-a03d-f2af6ab43b88 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:17.912315 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: WARNING py.warnings [None req-82a06eef-e2a9-4851-a03d-f2af6ab43b88 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 22:45:17.912315 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: warnings.warn(msg) Jul 20 22:45:17.912315 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: Jul 20 22:45:17.913587 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a06eef-e2a9-4851-a03d-f2af6ab43b88 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:17.995518 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 776/1557] 10.222.0.81 () {60 vars in 1143 bytes} [Wed Jul 20 22:45:17 2022] POST /identity/v3/users => generated 303 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:45:18.000449 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 782/1558] 10.222.0.81 () {60 vars in 1143 bytes} [Wed Jul 20 22:45:17 2022] POST /identity/v3/users => generated 288 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:45:18.000596 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367c9095-7ece-440f-8ae8-5537e9f19931 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:18.000698 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9416e169-ede8-4133-8fe0-9f9497d3ce9a None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:18.000698 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9416e169-ede8-4133-8fe0-9f9497d3ce9a None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:18.000866 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367c9095-7ece-440f-8ae8-5537e9f19931 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:18.000948 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9416e169-ede8-4133-8fe0-9f9497d3ce9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:18.001029 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367c9095-7ece-440f-8ae8-5537e9f19931 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:18.018395 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-9416e169-ede8-4133-8fe0-9f9497d3ce9a None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:45:18.018629 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-367c9095-7ece-440f-8ae8-5537e9f19931 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:45:18.019638 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-9416e169-ede8-4133-8fe0-9f9497d3ce9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:18.019761 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-367c9095-7ece-440f-8ae8-5537e9f19931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:18.027690 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 783/1559] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:45:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:45:18.031877 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 777/1560] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:45:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:45:18.032061 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9416e169-ede8-4133-8fe0-9f9497d3ce9a None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:18.032175 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3cba95-6404-4b30-93d1-44ee6ef2146e None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:18.032175 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3cba95-6404-4b30-93d1-44ee6ef2146e None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:18.032175 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3cba95-6404-4b30-93d1-44ee6ef2146e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:18.042517 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9416e169-ede8-4133-8fe0-9f9497d3ce9a None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:18.047401 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636e940c-7541-41f7-a877-ac33c2b83a58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:18.048056 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636e940c-7541-41f7-a877-ac33c2b83a58 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:18.048439 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636e940c-7541-41f7-a877-ac33c2b83a58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:18.049264 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636e940c-7541-41f7-a877-ac33c2b83a58 admin admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:18.049966 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636e940c-7541-41f7-a877-ac33c2b83a58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:18.050792 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-fb3cba95-6404-4b30-93d1-44ee6ef2146e None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:45:18.051943 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636e940c-7541-41f7-a877-ac33c2b83a58 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:18.052111 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-fb3cba95-6404-4b30-93d1-44ee6ef2146e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:18.060051 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 778/1561] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:45:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:45:18.060843 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3cba95-6404-4b30-93d1-44ee6ef2146e None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:18.070939 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3cba95-6404-4b30-93d1-44ee6ef2146e None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:18.075664 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe5ce41-6eb7-43d9-8745-f7b71134baa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:18.076321 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe5ce41-6eb7-43d9-8745-f7b71134baa5 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:18.076610 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe5ce41-6eb7-43d9-8745-f7b71134baa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:18.076895 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe5ce41-6eb7-43d9-8745-f7b71134baa5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:18.077619 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe5ce41-6eb7-43d9-8745-f7b71134baa5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:45:18.077932 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe5ce41-6eb7-43d9-8745-f7b71134baa5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:18.079564 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe5ce41-6eb7-43d9-8745-f7b71134baa5 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:18.087723 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:45:18.091521 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 779/1562] 10.222.0.81 () {58 vars in 1152 bytes} [Wed Jul 20 22:45:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:45:18.092822 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47114e19-aaaf-40e6-a16c-346023a68126 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:18.093119 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47114e19-aaaf-40e6-a16c-346023a68126 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:18.093405 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47114e19-aaaf-40e6-a16c-346023a68126 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:18.111519 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-47114e19-aaaf-40e6-a16c-346023a68126 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:45:18.116269 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-47114e19-aaaf-40e6-a16c-346023a68126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:18.120945 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 780/1563] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:45:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:45:18.121710 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47114e19-aaaf-40e6-a16c-346023a68126 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:18.130801 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47114e19-aaaf-40e6-a16c-346023a68126 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:18.135388 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df8a793-3ad2-4c1f-9a72-7b93347d8ecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:18.136025 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df8a793-3ad2-4c1f-9a72-7b93347d8ecb admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:18.136314 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df8a793-3ad2-4c1f-9a72-7b93347d8ecb admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:18.136596 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df8a793-3ad2-4c1f-9a72-7b93347d8ecb admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:18.137303 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df8a793-3ad2-4c1f-9a72-7b93347d8ecb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:18.138887 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df8a793-3ad2-4c1f-9a72-7b93347d8ecb admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:18.148706 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 781/1564] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:45:18 2022] GET /identity/v3/roles => generated 1420 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:45:18.152355 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df8a793-3ad2-4c1f-9a72-7b93347d8ecb admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:18.159706 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df8a793-3ad2-4c1f-9a72-7b93347d8ecb admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:18.164675 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f58ab10-6704-4017-b4fe-4643bdb1536a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:18.165344 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f58ab10-6704-4017-b4fe-4643bdb1536a admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:18.165663 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f58ab10-6704-4017-b4fe-4643bdb1536a admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:18.165945 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f58ab10-6704-4017-b4fe-4643bdb1536a admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:18.166661 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f58ab10-6704-4017-b4fe-4643bdb1536a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:18.168295 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f58ab10-6704-4017-b4fe-4643bdb1536a admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:18.179082 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 782/1565] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:45:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:45:18.179897 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f58ab10-6704-4017-b4fe-4643bdb1536a admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:18.185738 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 784/1566] 10.222.0.81 () {60 vars in 1152 bytes} [Wed Jul 20 22:45:18 2022] POST /identity/v3/projects => generated 385 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:45:18.185738 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636e940c-7541-41f7-a877-ac33c2b83a58 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:18.190504 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f58ab10-6704-4017-b4fe-4643bdb1536a admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:18.219172 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:45:18.246113 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1745364f-3ca6-4c04-b2b8-474061e2eae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:18.246693 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1745364f-3ca6-4c04-b2b8-474061e2eae5 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:18.246983 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1745364f-3ca6-4c04-b2b8-474061e2eae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:18.247288 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1745364f-3ca6-4c04-b2b8-474061e2eae5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:18.248000 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1745364f-3ca6-4c04-b2b8-474061e2eae5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:45:18.248307 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1745364f-3ca6-4c04-b2b8-474061e2eae5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:18.250096 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1745364f-3ca6-4c04-b2b8-474061e2eae5 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:18.252256 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636e940c-7541-41f7-a877-ac33c2b83a58 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:18.256966 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81681bcf-a03c-4663-9918-f91c3467ae97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:18.257835 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81681bcf-a03c-4663-9918-f91c3467ae97 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:18.258127 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81681bcf-a03c-4663-9918-f91c3467ae97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:18.258410 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81681bcf-a03c-4663-9918-f91c3467ae97 admin admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:18.259180 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81681bcf-a03c-4663-9918-f91c3467ae97 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:45:18.259515 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81681bcf-a03c-4663-9918-f91c3467ae97 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:18.261134 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81681bcf-a03c-4663-9918-f91c3467ae97 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:18.261960 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 783/1567] 10.222.0.81 () {58 vars in 1152 bytes} [Wed Jul 20 22:45:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:45:18.262767 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1745364f-3ca6-4c04-b2b8-474061e2eae5 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:18.272328 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1745364f-3ca6-4c04-b2b8-474061e2eae5 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:18.284837 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 785/1568] 10.222.0.81 () {58 vars in 1152 bytes} [Wed Jul 20 22:45:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:45:18.284837 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81681bcf-a03c-4663-9918-f91c3467ae97 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:18.285079 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ec4690-531e-4edc-b389-7b179ee2f579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:18.285079 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ec4690-531e-4edc-b389-7b179ee2f579 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:18.285079 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ec4690-531e-4edc-b389-7b179ee2f579 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:18.285079 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ec4690-531e-4edc-b389-7b179ee2f579 admin admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:18.285434 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ec4690-531e-4edc-b389-7b179ee2f579 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:45:18.285434 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ec4690-531e-4edc-b389-7b179ee2f579 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:18.285434 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ec4690-531e-4edc-b389-7b179ee2f579 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:18.285955 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81681bcf-a03c-4663-9918-f91c3467ae97 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:18.289214 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87957c46-5e43-498e-ad9f-46b060921170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:18.289859 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87957c46-5e43-498e-ad9f-46b060921170 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:18.290149 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87957c46-5e43-498e-ad9f-46b060921170 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:18.290433 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87957c46-5e43-498e-ad9f-46b060921170 admin admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:18.291154 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87957c46-5e43-498e-ad9f-46b060921170 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:18.293039 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87957c46-5e43-498e-ad9f-46b060921170 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:18.350334 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 784/1569] 10.222.0.81 () {58 vars in 1152 bytes} [Wed Jul 20 22:45:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:45:18.351346 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ec4690-531e-4edc-b389-7b179ee2f579 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:18.362537 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ec4690-531e-4edc-b389-7b179ee2f579 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:18.367242 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60663191-0d47-4c61-b093-16f326774f8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:18.367892 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60663191-0d47-4c61-b093-16f326774f8c admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:18.368177 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60663191-0d47-4c61-b093-16f326774f8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:18.368456 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60663191-0d47-4c61-b093-16f326774f8c admin admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:18.369300 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60663191-0d47-4c61-b093-16f326774f8c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:45:18.369610 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60663191-0d47-4c61-b093-16f326774f8c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:18.371220 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60663191-0d47-4c61-b093-16f326774f8c admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:18.384377 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 785/1570] 10.222.0.81 () {58 vars in 1152 bytes} [Wed Jul 20 22:45:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:45:18.385149 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60663191-0d47-4c61-b093-16f326774f8c admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:18.397419 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60663191-0d47-4c61-b093-16f326774f8c admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:18.402197 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca158aa-0abd-4c07-ba03-f3cf3e8ea420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:18.402907 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca158aa-0abd-4c07-ba03-f3cf3e8ea420 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:18.403395 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca158aa-0abd-4c07-ba03-f3cf3e8ea420 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:18.403690 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca158aa-0abd-4c07-ba03-f3cf3e8ea420 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:18.404426 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca158aa-0abd-4c07-ba03-f3cf3e8ea420 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:18.406077 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca158aa-0abd-4c07-ba03-f3cf3e8ea420 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:18.417961 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 786/1571] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:45:18 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:45:18.419618 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca158aa-0abd-4c07-ba03-f3cf3e8ea420 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:18.430165 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca158aa-0abd-4c07-ba03-f3cf3e8ea420 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:18.434929 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5e4273-807d-4f68-9cde-7dceab31cd94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:18.435656 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5e4273-807d-4f68-9cde-7dceab31cd94 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:18.435945 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5e4273-807d-4f68-9cde-7dceab31cd94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:18.436226 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5e4273-807d-4f68-9cde-7dceab31cd94 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:18.436995 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5e4273-807d-4f68-9cde-7dceab31cd94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:18.438619 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5e4273-807d-4f68-9cde-7dceab31cd94 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:18.450079 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 786/1572] 10.222.0.81 () {60 vars in 1152 bytes} [Wed Jul 20 22:45:18 2022] POST /identity/v3/projects => generated 379 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:45:18.450079 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87957c46-5e43-498e-ad9f-46b060921170 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:18.453539 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 787/1573] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:45:18 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:45:18.454374 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5e4273-807d-4f68-9cde-7dceab31cd94 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:18.465677 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5e4273-807d-4f68-9cde-7dceab31cd94 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:18.520880 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87957c46-5e43-498e-ad9f-46b060921170 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:18.525559 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a727493-c406-43f1-a2b1-575761e52447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:18.526262 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a727493-c406-43f1-a2b1-575761e52447 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:18.526542 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a727493-c406-43f1-a2b1-575761e52447 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:18.526844 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a727493-c406-43f1-a2b1-575761e52447 admin admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:18.527572 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a727493-c406-43f1-a2b1-575761e52447 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:18.528845 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5555ba6-c6ed-42f9-bb69-05d6b40e28ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:18.529482 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a727493-c406-43f1-a2b1-575761e52447 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:18.529592 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5555ba6-c6ed-42f9-bb69-05d6b40e28ab admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:18.529767 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5555ba6-c6ed-42f9-bb69-05d6b40e28ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:18.530046 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5555ba6-c6ed-42f9-bb69-05d6b40e28ab admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:18.530736 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5555ba6-c6ed-42f9-bb69-05d6b40e28ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:18.532755 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5555ba6-c6ed-42f9-bb69-05d6b40e28ab admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:18.547164 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:45:18.667530 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 787/1574] 10.222.0.81 () {60 vars in 1143 bytes} [Wed Jul 20 22:45:18 2022] POST /identity/v3/users => generated 293 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:45:18.668390 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a727493-c406-43f1-a2b1-575761e52447 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:18.677767 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a727493-c406-43f1-a2b1-575761e52447 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:18.682419 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6650bfc-b88b-460a-89b2-030f968c3040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:18.683092 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6650bfc-b88b-460a-89b2-030f968c3040 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:18.683379 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6650bfc-b88b-460a-89b2-030f968c3040 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:18.683652 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6650bfc-b88b-460a-89b2-030f968c3040 admin admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:18.684317 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6650bfc-b88b-460a-89b2-030f968c3040 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:18.686193 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6650bfc-b88b-460a-89b2-030f968c3040 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:18.698370 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 788/1575] 10.222.0.81 () {60 vars in 1152 bytes} [Wed Jul 20 22:45:18 2022] POST /identity/v3/projects => generated 391 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:45:18.699134 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5555ba6-c6ed-42f9-bb69-05d6b40e28ab admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:18.758028 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5555ba6-c6ed-42f9-bb69-05d6b40e28ab admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:18.765223 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f62e0ac-af5c-4a50-9a6a-da0c36e33ab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:18.765916 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f62e0ac-af5c-4a50-9a6a-da0c36e33ab5 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:18.766212 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f62e0ac-af5c-4a50-9a6a-da0c36e33ab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:18.766486 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f62e0ac-af5c-4a50-9a6a-da0c36e33ab5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:18.767780 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f62e0ac-af5c-4a50-9a6a-da0c36e33ab5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:18.769871 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f62e0ac-af5c-4a50-9a6a-da0c36e33ab5 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:18.794544 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 788/1576] 10.222.0.81 () {60 vars in 1152 bytes} [Wed Jul 20 22:45:18 2022] POST /identity/v3/projects => generated 405 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:45:18.796896 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6650bfc-b88b-460a-89b2-030f968c3040 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:18.856497 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6650bfc-b88b-460a-89b2-030f968c3040 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:18.861221 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbd40ea-abb3-4b9b-af65-a57331071008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:18.861885 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbd40ea-abb3-4b9b-af65-a57331071008 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:18.862220 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbd40ea-abb3-4b9b-af65-a57331071008 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:18.862507 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbd40ea-abb3-4b9b-af65-a57331071008 admin admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:18.863232 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbd40ea-abb3-4b9b-af65-a57331071008 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:18.863644 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 789/1577] 10.222.0.81 () {60 vars in 1152 bytes} [Wed Jul 20 22:45:18 2022] POST /identity/v3/projects => generated 381 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:45:18.865148 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbd40ea-abb3-4b9b-af65-a57331071008 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:18.865276 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f62e0ac-af5c-4a50-9a6a-da0c36e33ab5 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:18.875662 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:45:18.927272 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f62e0ac-af5c-4a50-9a6a-da0c36e33ab5 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:18.931949 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73163112-bdcd-4854-8954-dc829a6c15d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:18.932847 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73163112-bdcd-4854-8954-dc829a6c15d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:18.933131 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73163112-bdcd-4854-8954-dc829a6c15d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:18.933406 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73163112-bdcd-4854-8954-dc829a6c15d0 admin admin] PATH_INFO: `/v3/projects/8467d6d569584e26b7c4e1ceb3126598/users/3cd5c46e50714155aed3e9f439c2bf6d/roles/559942c4de1744169f8b5e0404ecde6b` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:18.948117 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73163112-bdcd-4854-8954-dc829a6c15d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8467d6d569584e26b7c4e1ceb3126598, user_id=3cd5c46e50714155aed3e9f439c2bf6d, role_id=559942c4de1744169f8b5e0404ecde6b)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:18.949159 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: WARNING py.warnings [None req-73163112-bdcd-4854-8954-dc829a6c15d0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 22:45:18.949159 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: warnings.warn(msg) Jul 20 22:45:18.949159 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: Jul 20 22:45:18.951193 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73163112-bdcd-4854-8954-dc829a6c15d0 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:18.988778 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 789/1578] 10.222.0.81 () {60 vars in 1152 bytes} [Wed Jul 20 22:45:18 2022] POST /identity/v3/projects => generated 385 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:45:18.989717 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbd40ea-abb3-4b9b-af65-a57331071008 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:19.022728 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 790/1579] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:45:18 2022] PUT /identity/v3/projects/8467d6d569584e26b7c4e1ceb3126598/users/3cd5c46e50714155aed3e9f439c2bf6d/roles/559942c4de1744169f8b5e0404ecde6b => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:45:19.023827 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73163112-bdcd-4854-8954-dc829a6c15d0 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:19.055867 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbd40ea-abb3-4b9b-af65-a57331071008 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:19.085734 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73163112-bdcd-4854-8954-dc829a6c15d0 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:19.090626 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5426e754-f31b-43e6-bb8c-4c15945acdd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:19.091588 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5426e754-f31b-43e6-bb8c-4c15945acdd6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:19.091875 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5426e754-f31b-43e6-bb8c-4c15945acdd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:19.092154 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5426e754-f31b-43e6-bb8c-4c15945acdd6 admin admin] PATH_INFO: `/v3/projects/90ac023afcb44b5097539298076c0838/users/0cf649be0c48492f8926f4436cbb3d6d/roles/559942c4de1744169f8b5e0404ecde6b` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:19.099909 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:45:19.108631 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5426e754-f31b-43e6-bb8c-4c15945acdd6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90ac023afcb44b5097539298076c0838, user_id=0cf649be0c48492f8926f4436cbb3d6d, role_id=559942c4de1744169f8b5e0404ecde6b)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:19.112519 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5426e754-f31b-43e6-bb8c-4c15945acdd6 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:19.119765 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24f20b9-e061-47ce-a231-4ea6665a9db9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:19.120481 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24f20b9-e061-47ce-a231-4ea6665a9db9 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:19.120789 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24f20b9-e061-47ce-a231-4ea6665a9db9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:19.121074 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24f20b9-e061-47ce-a231-4ea6665a9db9 admin admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:19.121913 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24f20b9-e061-47ce-a231-4ea6665a9db9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:19.123878 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24f20b9-e061-47ce-a231-4ea6665a9db9 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:19.211448 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 791/1580] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:45:19 2022] PUT /identity/v3/projects/90ac023afcb44b5097539298076c0838/users/0cf649be0c48492f8926f4436cbb3d6d/roles/559942c4de1744169f8b5e0404ecde6b => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:45:19.212314 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5426e754-f31b-43e6-bb8c-4c15945acdd6 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:19.225519 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 790/1581] 10.222.0.81 () {60 vars in 1143 bytes} [Wed Jul 20 22:45:18 2022] POST /identity/v3/users => generated 290 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:45:19.225519 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24f20b9-e061-47ce-a231-4ea6665a9db9 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:19.233381 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24f20b9-e061-47ce-a231-4ea6665a9db9 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:19.276050 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5426e754-f31b-43e6-bb8c-4c15945acdd6 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:19.281266 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecbb430-4290-439e-8764-13a5b4847dc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:19.281949 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecbb430-4290-439e-8764-13a5b4847dc0 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:19.282295 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecbb430-4290-439e-8764-13a5b4847dc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:19.282618 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecbb430-4290-439e-8764-13a5b4847dc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:19.283969 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecbb430-4290-439e-8764-13a5b4847dc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:19.285814 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecbb430-4290-439e-8764-13a5b4847dc0 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:19.296308 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 792/1582] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:45:19 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:45:19.296464 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3931fc3d-6bc2-4581-89dc-a1b92102fc35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:19.297024 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3931fc3d-6bc2-4581-89dc-a1b92102fc35 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:19.297174 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecbb430-4290-439e-8764-13a5b4847dc0 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:19.297310 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3931fc3d-6bc2-4581-89dc-a1b92102fc35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:19.297596 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3931fc3d-6bc2-4581-89dc-a1b92102fc35 admin admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:19.298293 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3931fc3d-6bc2-4581-89dc-a1b92102fc35 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:19.300253 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3931fc3d-6bc2-4581-89dc-a1b92102fc35 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:19.307459 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecbb430-4290-439e-8764-13a5b4847dc0 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:19.311812 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac75cc5-1d0f-4ff5-86e0-53cb7b649962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:19.312478 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac75cc5-1d0f-4ff5-86e0-53cb7b649962 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:19.312767 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac75cc5-1d0f-4ff5-86e0-53cb7b649962 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:19.313049 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac75cc5-1d0f-4ff5-86e0-53cb7b649962 admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:19.313735 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac75cc5-1d0f-4ff5-86e0-53cb7b649962 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:19.315704 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac75cc5-1d0f-4ff5-86e0-53cb7b649962 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:19.375809 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 791/1583] 10.222.0.81 () {60 vars in 1143 bytes} [Wed Jul 20 22:45:19 2022] POST /identity/v3/users => generated 296 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:45:19.376790 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3931fc3d-6bc2-4581-89dc-a1b92102fc35 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:19.386524 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3931fc3d-6bc2-4581-89dc-a1b92102fc35 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:19.391287 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290ecfe5-ed9c-43f7-90b8-2de55d7e7e3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:19.391940 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290ecfe5-ed9c-43f7-90b8-2de55d7e7e3f admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:19.392999 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290ecfe5-ed9c-43f7-90b8-2de55d7e7e3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:19.393427 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290ecfe5-ed9c-43f7-90b8-2de55d7e7e3f admin admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:19.397198 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 793/1584] 10.222.0.81 () {60 vars in 1143 bytes} [Wed Jul 20 22:45:19 2022] POST /identity/v3/users => generated 303 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:45:19.397360 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290ecfe5-ed9c-43f7-90b8-2de55d7e7e3f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:19.397460 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac75cc5-1d0f-4ff5-86e0-53cb7b649962 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:19.397561 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290ecfe5-ed9c-43f7-90b8-2de55d7e7e3f admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:19.405044 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac75cc5-1d0f-4ff5-86e0-53cb7b649962 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:19.411701 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f0d181-cf79-40a2-8995-7379aac4273e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:19.412358 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f0d181-cf79-40a2-8995-7379aac4273e admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:19.412648 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f0d181-cf79-40a2-8995-7379aac4273e admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:19.412930 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f0d181-cf79-40a2-8995-7379aac4273e admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:19.413612 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f0d181-cf79-40a2-8995-7379aac4273e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:19.415564 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f0d181-cf79-40a2-8995-7379aac4273e admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:19.486879 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:45:19.492313 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 792/1585] 10.222.0.81 () {60 vars in 1143 bytes} [Wed Jul 20 22:45:19 2022] POST /identity/v3/users => generated 291 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:45:19.493750 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7eb48d-9921-4521-8e79-e00ef7f8a341 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:19.494066 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7eb48d-9921-4521-8e79-e00ef7f8a341 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:19.494333 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7eb48d-9921-4521-8e79-e00ef7f8a341 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:19.511961 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-ba7eb48d-9921-4521-8e79-e00ef7f8a341 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:45:19.513198 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-ba7eb48d-9921-4521-8e79-e00ef7f8a341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:19.516794 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 794/1586] 10.222.0.81 () {60 vars in 1143 bytes} [Wed Jul 20 22:45:19 2022] POST /identity/v3/users => generated 293 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:45:19.516794 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277ce648-6b7f-440b-bd9d-f702bfdc54d7 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:19.516794 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277ce648-6b7f-440b-bd9d-f702bfdc54d7 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:19.516794 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277ce648-6b7f-440b-bd9d-f702bfdc54d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:19.524116 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 793/1587] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:45:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:45:19.524875 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7eb48d-9921-4521-8e79-e00ef7f8a341 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:19.534448 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-277ce648-6b7f-440b-bd9d-f702bfdc54d7 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:45:19.534683 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7eb48d-9921-4521-8e79-e00ef7f8a341 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:19.536006 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-277ce648-6b7f-440b-bd9d-f702bfdc54d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:19.540025 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef786ac8-35e7-4baa-ae97-c0661022fe34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:19.540529 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef786ac8-35e7-4baa-ae97-c0661022fe34 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:19.540818 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef786ac8-35e7-4baa-ae97-c0661022fe34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:19.541103 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef786ac8-35e7-4baa-ae97-c0661022fe34 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:19.541832 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef786ac8-35e7-4baa-ae97-c0661022fe34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:19.543481 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef786ac8-35e7-4baa-ae97-c0661022fe34 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:19.543854 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 795/1588] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:45:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:45:19.544632 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277ce648-6b7f-440b-bd9d-f702bfdc54d7 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:19.553780 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277ce648-6b7f-440b-bd9d-f702bfdc54d7 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:19.554714 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 794/1589] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:45:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:45:19.555593 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef786ac8-35e7-4baa-ae97-c0661022fe34 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:19.559266 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753c3220-f058-4833-8a1e-6cd02a6ded83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:19.559884 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753c3220-f058-4833-8a1e-6cd02a6ded83 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:19.560170 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753c3220-f058-4833-8a1e-6cd02a6ded83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:19.560451 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753c3220-f058-4833-8a1e-6cd02a6ded83 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:19.561183 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753c3220-f058-4833-8a1e-6cd02a6ded83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:19.562949 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753c3220-f058-4833-8a1e-6cd02a6ded83 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:19.566559 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef786ac8-35e7-4baa-ae97-c0661022fe34 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:19.571291 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3364bac-2565-4bda-8477-d7d65548d957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:19.571947 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3364bac-2565-4bda-8477-d7d65548d957 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:19.572235 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3364bac-2565-4bda-8477-d7d65548d957 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:19.572518 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3364bac-2565-4bda-8477-d7d65548d957 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:19.573237 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3364bac-2565-4bda-8477-d7d65548d957 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:19.576724 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 796/1590] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:45:19 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:45:19.576724 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753c3220-f058-4833-8a1e-6cd02a6ded83 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:19.576941 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3364bac-2565-4bda-8477-d7d65548d957 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:19.586725 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753c3220-f058-4833-8a1e-6cd02a6ded83 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:19.587057 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 795/1591] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:45:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:45:19.588086 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3364bac-2565-4bda-8477-d7d65548d957 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:19.591608 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7467f965-48e8-4e84-9058-94e21e3874fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:19.592278 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7467f965-48e8-4e84-9058-94e21e3874fe admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:19.592564 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7467f965-48e8-4e84-9058-94e21e3874fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:19.592865 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7467f965-48e8-4e84-9058-94e21e3874fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:19.593608 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7467f965-48e8-4e84-9058-94e21e3874fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:19.595971 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7467f965-48e8-4e84-9058-94e21e3874fe admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:19.598598 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3364bac-2565-4bda-8477-d7d65548d957 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:19.603556 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9b23be-f84a-452f-846e-fd0f355c90e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:19.604199 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9b23be-f84a-452f-846e-fd0f355c90e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:19.604487 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9b23be-f84a-452f-846e-fd0f355c90e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:19.604768 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9b23be-f84a-452f-846e-fd0f355c90e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:19.605486 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9b23be-f84a-452f-846e-fd0f355c90e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:19.606140 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 797/1592] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:45:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:45:19.606995 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7467f965-48e8-4e84-9058-94e21e3874fe admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:19.607144 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9b23be-f84a-452f-846e-fd0f355c90e0 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:19.620370 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 796/1593] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:45:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:45:19.620549 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7467f965-48e8-4e84-9058-94e21e3874fe admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:19.620663 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9b23be-f84a-452f-846e-fd0f355c90e0 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:19.623029 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc72eb49-cd2b-4e09-8523-350a6388464b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:19.623698 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc72eb49-cd2b-4e09-8523-350a6388464b admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:19.623985 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc72eb49-cd2b-4e09-8523-350a6388464b admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:19.624264 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc72eb49-cd2b-4e09-8523-350a6388464b admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:19.624980 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc72eb49-cd2b-4e09-8523-350a6388464b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:19.627123 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc72eb49-cd2b-4e09-8523-350a6388464b admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:19.637823 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 798/1594] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:45:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:45:19.638900 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc72eb49-cd2b-4e09-8523-350a6388464b admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:19.648942 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc72eb49-cd2b-4e09-8523-350a6388464b admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:19.653456 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0918e6f-9cc3-4071-907f-fa2b4df39881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:19.654087 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0918e6f-9cc3-4071-907f-fa2b4df39881 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:19.654376 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0918e6f-9cc3-4071-907f-fa2b4df39881 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:19.654653 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0918e6f-9cc3-4071-907f-fa2b4df39881 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:19.655386 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0918e6f-9cc3-4071-907f-fa2b4df39881 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:19.656936 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0918e6f-9cc3-4071-907f-fa2b4df39881 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:19.667352 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 799/1595] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:45:19 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:45:19.668134 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0918e6f-9cc3-4071-907f-fa2b4df39881 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:19.676486 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9b23be-f84a-452f-846e-fd0f355c90e0 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:19.676976 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0918e6f-9cc3-4071-907f-fa2b4df39881 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:19.680263 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f4615a-dc18-432d-a5cb-5e1bbb1a4e25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:19.680548 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347dadf6-1636-40b0-88b0-e3b031d0a172 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:19.680914 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f4615a-dc18-432d-a5cb-5e1bbb1a4e25 service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:19.681160 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347dadf6-1636-40b0-88b0-e3b031d0a172 service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:19.681274 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f4615a-dc18-432d-a5cb-5e1bbb1a4e25 service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:19.681445 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347dadf6-1636-40b0-88b0-e3b031d0a172 service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:19.681550 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f4615a-dc18-432d-a5cb-5e1bbb1a4e25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:19.681729 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347dadf6-1636-40b0-88b0-e3b031d0a172 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:19.691083 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347dadf6-1636-40b0-88b0-e3b031d0a172 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:19.691265 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f4615a-dc18-432d-a5cb-5e1bbb1a4e25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:19.692519 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347dadf6-1636-40b0-88b0-e3b031d0a172 service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:19.692744 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f4615a-dc18-432d-a5cb-5e1bbb1a4e25 service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:19.702228 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 800/1596] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:45:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:45:19.702579 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 797/1597] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:45:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:45:19.703126 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347dadf6-1636-40b0-88b0-e3b031d0a172 service neutron] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:19.703416 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f4615a-dc18-432d-a5cb-5e1bbb1a4e25 service neutron] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:19.712645 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347dadf6-1636-40b0-88b0-e3b031d0a172 service neutron] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:19.713859 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f4615a-dc18-432d-a5cb-5e1bbb1a4e25 service neutron] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:19.717352 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0cb7c0-0535-4320-989c-04317760a285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:19.718243 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0cb7c0-0535-4320-989c-04317760a285 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:19.718529 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0cb7c0-0535-4320-989c-04317760a285 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:19.718632 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8badc9c7-2027-45dd-a9ed-3be8a3484a5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:19.718822 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0cb7c0-0535-4320-989c-04317760a285 admin admin] PATH_INFO: `/v3/projects/10dcd3a5465d434cbb63e588e4d4df8c/users/906771f309044b178b9218d79390e35d/roles/559942c4de1744169f8b5e0404ecde6b` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:19.719358 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8badc9c7-2027-45dd-a9ed-3be8a3484a5f admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:19.719643 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8badc9c7-2027-45dd-a9ed-3be8a3484a5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:19.719947 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8badc9c7-2027-45dd-a9ed-3be8a3484a5f admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:19.720667 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8badc9c7-2027-45dd-a9ed-3be8a3484a5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:19.722314 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8badc9c7-2027-45dd-a9ed-3be8a3484a5f admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:19.734065 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 798/1598] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:45:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:45:19.734328 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0cb7c0-0535-4320-989c-04317760a285 admin admin] RBAC: Authorizing `identity:create_grant(project_id=10dcd3a5465d434cbb63e588e4d4df8c, user_id=906771f309044b178b9218d79390e35d, role_id=559942c4de1744169f8b5e0404ecde6b)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:19.737054 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0cb7c0-0535-4320-989c-04317760a285 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:19.738068 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8badc9c7-2027-45dd-a9ed-3be8a3484a5f admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:19.748907 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8badc9c7-2027-45dd-a9ed-3be8a3484a5f admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:19.754362 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787a5bbc-0168-4e3e-a8d2-f8a7a8dddb1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:19.755079 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787a5bbc-0168-4e3e-a8d2-f8a7a8dddb1f admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:19.755371 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787a5bbc-0168-4e3e-a8d2-f8a7a8dddb1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:19.755653 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787a5bbc-0168-4e3e-a8d2-f8a7a8dddb1f admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:19.756375 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787a5bbc-0168-4e3e-a8d2-f8a7a8dddb1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:19.758044 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787a5bbc-0168-4e3e-a8d2-f8a7a8dddb1f admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:19.771696 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 799/1599] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:45:19 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:45:19.771696 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787a5bbc-0168-4e3e-a8d2-f8a7a8dddb1f admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:19.778661 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787a5bbc-0168-4e3e-a8d2-f8a7a8dddb1f admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:19.783839 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47253e3c-b85b-4cc2-be34-8fececaf54f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:19.786167 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47253e3c-b85b-4cc2-be34-8fececaf54f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:19.786167 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47253e3c-b85b-4cc2-be34-8fececaf54f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:19.786167 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47253e3c-b85b-4cc2-be34-8fececaf54f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:19.786167 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47253e3c-b85b-4cc2-be34-8fececaf54f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:19.787718 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47253e3c-b85b-4cc2-be34-8fececaf54f7 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:19.798196 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 800/1600] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:45:19 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:45:19.799573 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47253e3c-b85b-4cc2-be34-8fececaf54f7 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:19.809128 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47253e3c-b85b-4cc2-be34-8fececaf54f7 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:19.811293 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 801/1601] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:45:19 2022] PUT /identity/v3/projects/10dcd3a5465d434cbb63e588e4d4df8c/users/906771f309044b178b9218d79390e35d/roles/559942c4de1744169f8b5e0404ecde6b => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:45:19.812276 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0cb7c0-0535-4320-989c-04317760a285 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:19.844546 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:45:19.868021 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29b170a-7091-454b-afb4-7c989f10e49b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:19.870264 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29b170a-7091-454b-afb4-7c989f10e49b admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:19.870264 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29b170a-7091-454b-afb4-7c989f10e49b admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:19.870264 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29b170a-7091-454b-afb4-7c989f10e49b admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:19.870264 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29b170a-7091-454b-afb4-7c989f10e49b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:19.871816 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29b170a-7091-454b-afb4-7c989f10e49b admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:19.885931 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 801/1602] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:45:19 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:45:19.885931 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29b170a-7091-454b-afb4-7c989f10e49b admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:19.891060 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0cb7c0-0535-4320-989c-04317760a285 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:19.896585 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29b170a-7091-454b-afb4-7c989f10e49b admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:19.898488 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfbc936-2e2a-4c3c-920e-59cc9ce46bac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:19.901114 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfbc936-2e2a-4c3c-920e-59cc9ce46bac admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:19.901114 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfbc936-2e2a-4c3c-920e-59cc9ce46bac admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:19.901114 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfbc936-2e2a-4c3c-920e-59cc9ce46bac admin admin] PATH_INFO: `/v3/projects/788c1d44ad5d442e8ec58f1d76ae649b/users/1b458e3837be425bba1f880895918287/roles/559942c4de1744169f8b5e0404ecde6b` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:19.901934 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8537e1-78e3-4352-b0ac-c1e9c8766b6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:19.902944 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8537e1-78e3-4352-b0ac-c1e9c8766b6e admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:19.903269 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8537e1-78e3-4352-b0ac-c1e9c8766b6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:19.903553 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8537e1-78e3-4352-b0ac-c1e9c8766b6e admin admin] PATH_INFO: `/v3/projects/55c688e93c2b414ca86a3c2f9ec84798/users/1ed9249b2a9b42d9972ed71c04f2c888/roles/559942c4de1744169f8b5e0404ecde6b` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:19.916721 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfbc936-2e2a-4c3c-920e-59cc9ce46bac admin admin] RBAC: Authorizing `identity:create_grant(project_id=788c1d44ad5d442e8ec58f1d76ae649b, user_id=1b458e3837be425bba1f880895918287, role_id=559942c4de1744169f8b5e0404ecde6b)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:19.920294 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfbc936-2e2a-4c3c-920e-59cc9ce46bac admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:19.924752 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8537e1-78e3-4352-b0ac-c1e9c8766b6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=55c688e93c2b414ca86a3c2f9ec84798, user_id=1ed9249b2a9b42d9972ed71c04f2c888, role_id=559942c4de1744169f8b5e0404ecde6b)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:19.926847 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8537e1-78e3-4352-b0ac-c1e9c8766b6e admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:20.004104 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 802/1603] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:45:19 2022] PUT /identity/v3/projects/788c1d44ad5d442e8ec58f1d76ae649b/users/1b458e3837be425bba1f880895918287/roles/559942c4de1744169f8b5e0404ecde6b => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:45:20.004104 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfbc936-2e2a-4c3c-920e-59cc9ce46bac admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:20.004854 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 802/1604] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:45:19 2022] PUT /identity/v3/projects/55c688e93c2b414ca86a3c2f9ec84798/users/1ed9249b2a9b42d9972ed71c04f2c888/roles/559942c4de1744169f8b5e0404ecde6b => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:45:20.005677 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8537e1-78e3-4352-b0ac-c1e9c8766b6e admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:20.013671 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:45:20.076651 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8537e1-78e3-4352-b0ac-c1e9c8766b6e admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:20.080433 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfbc936-2e2a-4c3c-920e-59cc9ce46bac admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:20.082266 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b91edd8-cf50-4229-95a7-b3c536e89637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:20.083305 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b91edd8-cf50-4229-95a7-b3c536e89637 admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:20.083671 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b91edd8-cf50-4229-95a7-b3c536e89637 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:20.084015 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b91edd8-cf50-4229-95a7-b3c536e89637 admin admin] PATH_INFO: `/v3/projects/f733781536584f80acfc325bf8444536/users/3e072e59bf3e46aaa062ec85b0e2c91a/roles/559942c4de1744169f8b5e0404ecde6b` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:20.086468 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c7ba6d-7479-4e75-8bea-10ccbe31b535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:20.087532 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c7ba6d-7479-4e75-8bea-10ccbe31b535 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:20.087824 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c7ba6d-7479-4e75-8bea-10ccbe31b535 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:20.088109 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c7ba6d-7479-4e75-8bea-10ccbe31b535 admin admin] PATH_INFO: `/v3/projects/a3f935addf9a4578871c37310f0eb093/users/3d921323545e4762bd5cb003c5f4852c/roles/559942c4de1744169f8b5e0404ecde6b` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:20.102938 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b91edd8-cf50-4229-95a7-b3c536e89637 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f733781536584f80acfc325bf8444536, user_id=3e072e59bf3e46aaa062ec85b0e2c91a, role_id=559942c4de1744169f8b5e0404ecde6b)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:20.105443 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c7ba6d-7479-4e75-8bea-10ccbe31b535 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3f935addf9a4578871c37310f0eb093, user_id=3d921323545e4762bd5cb003c5f4852c, role_id=559942c4de1744169f8b5e0404ecde6b)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:20.105836 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b91edd8-cf50-4229-95a7-b3c536e89637 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:20.108121 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c7ba6d-7479-4e75-8bea-10ccbe31b535 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:20.226907 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 803/1605] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:45:20 2022] PUT /identity/v3/projects/f733781536584f80acfc325bf8444536/users/3e072e59bf3e46aaa062ec85b0e2c91a/roles/559942c4de1744169f8b5e0404ecde6b => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:45:20.226907 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5261238-5ba5-4c88-bc50-fd82eb1421e0 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:20.226907 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5261238-5ba5-4c88-bc50-fd82eb1421e0 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:20.226907 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5261238-5ba5-4c88-bc50-fd82eb1421e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:20.229298 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 803/1606] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:45:20 2022] PUT /identity/v3/projects/a3f935addf9a4578871c37310f0eb093/users/3d921323545e4762bd5cb003c5f4852c/roles/559942c4de1744169f8b5e0404ecde6b => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:45:20.232908 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c7ba6d-7479-4e75-8bea-10ccbe31b535 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:20.256240 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-f5261238-5ba5-4c88-bc50-fd82eb1421e0 None None] MFA Rules not processed for user `906771f309044b178b9218d79390e35d`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:45:20.257542 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-f5261238-5ba5-4c88-bc50-fd82eb1421e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:20.317327 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c7ba6d-7479-4e75-8bea-10ccbe31b535 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:20.349315 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 804/1607] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:45:20 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:45:20.350779 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d05156-aa28-47e2-90ba-784fb29bf99a None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:20.351226 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d05156-aa28-47e2-90ba-784fb29bf99a None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:20.352176 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d05156-aa28-47e2-90ba-784fb29bf99a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:20.356947 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:45:20.373149 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-96d05156-aa28-47e2-90ba-784fb29bf99a None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:45:20.374542 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-96d05156-aa28-47e2-90ba-784fb29bf99a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:20.399339 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef58c97b-713e-4830-90c7-71226070f52e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:20.400955 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef58c97b-713e-4830-90c7-71226070f52e admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:20.401390 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef58c97b-713e-4830-90c7-71226070f52e admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:20.402133 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef58c97b-713e-4830-90c7-71226070f52e admin admin] PATH_INFO: `/v3/projects/7c71439943d84ea888b6b20f2d78a0cf/users/69b8faa54ac144178b3a00b40ce7bf0d/roles/559942c4de1744169f8b5e0404ecde6b` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:20.420144 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef58c97b-713e-4830-90c7-71226070f52e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c71439943d84ea888b6b20f2d78a0cf, user_id=69b8faa54ac144178b3a00b40ce7bf0d, role_id=559942c4de1744169f8b5e0404ecde6b)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:20.422990 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef58c97b-713e-4830-90c7-71226070f52e admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:20.440901 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 805/1608] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:45:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:45:20.442182 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a21fb83-0627-44f8-9dae-86670b88d2c9 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:20.442475 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a21fb83-0627-44f8-9dae-86670b88d2c9 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:20.442754 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a21fb83-0627-44f8-9dae-86670b88d2c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:20.462671 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-3a21fb83-0627-44f8-9dae-86670b88d2c9 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:45:20.463919 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-3a21fb83-0627-44f8-9dae-86670b88d2c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:20.472491 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 806/1609] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:45:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:45:20.476343 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba6fd70-58da-4f30-9b2f-aace00b56fec None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:20.476966 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba6fd70-58da-4f30-9b2f-aace00b56fec None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:20.477957 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba6fd70-58da-4f30-9b2f-aace00b56fec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:20.500490 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-fba6fd70-58da-4f30-9b2f-aace00b56fec None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:45:20.502320 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-fba6fd70-58da-4f30-9b2f-aace00b56fec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:20.510083 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 807/1610] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:45:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:45:20.511393 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94cc8d3-ad28-4552-9ee5-9dc6b5ab83ea None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:20.511677 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94cc8d3-ad28-4552-9ee5-9dc6b5ab83ea None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:20.511958 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94cc8d3-ad28-4552-9ee5-9dc6b5ab83ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:20.519051 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:45:20.543320 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-b94cc8d3-ad28-4552-9ee5-9dc6b5ab83ea None None] MFA Rules not processed for user `3d921323545e4762bd5cb003c5f4852c`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:45:20.544718 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-b94cc8d3-ad28-4552-9ee5-9dc6b5ab83ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:20.569025 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 804/1611] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:45:20 2022] PUT /identity/v3/projects/7c71439943d84ea888b6b20f2d78a0cf/users/69b8faa54ac144178b3a00b40ce7bf0d/roles/559942c4de1744169f8b5e0404ecde6b => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:45:20.571226 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ad6ab4-45b4-4457-b0c1-4e3cbad08ed9 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:20.572058 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ad6ab4-45b4-4457-b0c1-4e3cbad08ed9 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:20.572348 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ad6ab4-45b4-4457-b0c1-4e3cbad08ed9 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:20.574719 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 805/1612] 10.222.0.81 () {54 vars in 962 bytes} [Wed Jul 20 22:45:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:45:20.575679 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ad6ab4-45b4-4457-b0c1-4e3cbad08ed9 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:20.647839 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ad6ab4-45b4-4457-b0c1-4e3cbad08ed9 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:20.652369 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460376e9-9d33-41ad-9d32-7d095f7f2aae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:20.654217 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460376e9-9d33-41ad-9d32-7d095f7f2aae service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:20.654570 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460376e9-9d33-41ad-9d32-7d095f7f2aae service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:20.654858 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460376e9-9d33-41ad-9d32-7d095f7f2aae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:20.666161 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 808/1613] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:45:20 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:45:20.670424 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94cc8d3-ad28-4552-9ee5-9dc6b5ab83ea None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:20.726760 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460376e9-9d33-41ad-9d32-7d095f7f2aae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:20.728244 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460376e9-9d33-41ad-9d32-7d095f7f2aae service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:20.733658 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94cc8d3-ad28-4552-9ee5-9dc6b5ab83ea None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:20.737945 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6ee007-e79a-46db-ac2e-6e7e3df1f019 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:20.738417 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 806/1614] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:45:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:45:20.738672 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6ee007-e79a-46db-ac2e-6e7e3df1f019 service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:20.738995 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6ee007-e79a-46db-ac2e-6e7e3df1f019 service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:20.739311 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6ee007-e79a-46db-ac2e-6e7e3df1f019 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:20.740107 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460376e9-9d33-41ad-9d32-7d095f7f2aae service neutron] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:20.741587 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-460376e9-9d33-41ad-9d32-7d095f7f2aae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:20.750035 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6ee007-e79a-46db-ac2e-6e7e3df1f019 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:20.752151 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6ee007-e79a-46db-ac2e-6e7e3df1f019 service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:20.754905 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460376e9-9d33-41ad-9d32-7d095f7f2aae service neutron] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:20.760011 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afce82a-82e2-4d2c-b3b6-ecb6b9936c92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:20.760722 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afce82a-82e2-4d2c-b3b6-ecb6b9936c92 service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:20.761009 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afce82a-82e2-4d2c-b3b6-ecb6b9936c92 service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:20.761288 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afce82a-82e2-4d2c-b3b6-ecb6b9936c92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:20.762304 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 809/1615] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:45:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:45:20.763617 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e82255-e723-4172-96d3-3933e3f28d24 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:20.763908 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e82255-e723-4172-96d3-3933e3f28d24 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:20.764310 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e82255-e723-4172-96d3-3933e3f28d24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:20.772574 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afce82a-82e2-4d2c-b3b6-ecb6b9936c92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:20.774407 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afce82a-82e2-4d2c-b3b6-ecb6b9936c92 service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:20.787351 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 807/1616] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:45:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:45:20.788788 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac6d784-ef4e-4c8c-81f3-3ee4852f9849 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:20.789088 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac6d784-ef4e-4c8c-81f3-3ee4852f9849 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:20.789373 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac6d784-ef4e-4c8c-81f3-3ee4852f9849 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:20.793359 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-72e82255-e723-4172-96d3-3933e3f28d24 None None] MFA Rules not processed for user `69b8faa54ac144178b3a00b40ce7bf0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:45:20.794632 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-72e82255-e723-4172-96d3-3933e3f28d24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:20.827929 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-dac6d784-ef4e-4c8c-81f3-3ee4852f9849 None None] MFA Rules not processed for user `7829324db2574d38b18c749d0559177f`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:45:20.829167 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-dac6d784-ef4e-4c8c-81f3-3ee4852f9849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:20.862083 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 810/1617] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:45:20 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:45:20.867735 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e82255-e723-4172-96d3-3933e3f28d24 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:20.874201 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e82255-e723-4172-96d3-3933e3f28d24 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:20.880379 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967119e5-41c7-486f-8bbd-3c3eb474a0c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:20.881062 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967119e5-41c7-486f-8bbd-3c3eb474a0c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:20.881416 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967119e5-41c7-486f-8bbd-3c3eb474a0c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:20.881704 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967119e5-41c7-486f-8bbd-3c3eb474a0c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:20.882422 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967119e5-41c7-486f-8bbd-3c3eb474a0c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:20.884434 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967119e5-41c7-486f-8bbd-3c3eb474a0c9 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:20.905807 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 808/1618] 10.222.0.81 () {58 vars in 1059 bytes} [Wed Jul 20 22:45:20 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:45:20.909669 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac6d784-ef4e-4c8c-81f3-3ee4852f9849 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:20.919333 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac6d784-ef4e-4c8c-81f3-3ee4852f9849 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:20.924557 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0950ad3-5f5a-4eac-a4d6-20e96c6075ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:20.925267 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0950ad3-5f5a-4eac-a4d6-20e96c6075ff admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:20.925553 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0950ad3-5f5a-4eac-a4d6-20e96c6075ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:20.925831 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0950ad3-5f5a-4eac-a4d6-20e96c6075ff admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:20.926519 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0950ad3-5f5a-4eac-a4d6-20e96c6075ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:20.928798 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0950ad3-5f5a-4eac-a4d6-20e96c6075ff admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:21.073173 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 811/1619] 10.222.0.81 () {60 vars in 1152 bytes} [Wed Jul 20 22:45:20 2022] POST /identity/v3/projects => generated 405 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:45:21.073173 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967119e5-41c7-486f-8bbd-3c3eb474a0c9 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:21.103458 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:45:21.113334 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 809/1620] 10.222.0.81 () {60 vars in 1152 bytes} [Wed Jul 20 22:45:20 2022] POST /identity/v3/projects => generated 387 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:45:21.113334 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0950ad3-5f5a-4eac-a4d6-20e96c6075ff admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:21.200485 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967119e5-41c7-486f-8bbd-3c3eb474a0c9 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:21.205690 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0950ad3-5f5a-4eac-a4d6-20e96c6075ff admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:21.210442 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f10b92-6b4a-4c85-bfa1-903c4f4d7fa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:21.211183 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f10b92-6b4a-4c85-bfa1-903c4f4d7fa4 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:21.211475 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f10b92-6b4a-4c85-bfa1-903c4f4d7fa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:21.211754 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f10b92-6b4a-4c85-bfa1-903c4f4d7fa4 admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:21.212448 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f10b92-6b4a-4c85-bfa1-903c4f4d7fa4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:21.214443 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f10b92-6b4a-4c85-bfa1-903c4f4d7fa4 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:21.270790 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0eb08a4-52fb-4ea6-9c9c-6e9ac7ddea2e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7829324db2574d38b18c749d0559177f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:21.271488 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eb08a4-52fb-4ea6-9c9c-6e9ac7ddea2e service nova] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:21.271822 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eb08a4-52fb-4ea6-9c9c-6e9ac7ddea2e service nova] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:21.272107 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eb08a4-52fb-4ea6-9c9c-6e9ac7ddea2e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:21.328470 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0eb08a4-52fb-4ea6-9c9c-6e9ac7ddea2e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:21.330610 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0eb08a4-52fb-4ea6-9c9c-6e9ac7ddea2e service nova] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:21.339681 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 812/1621] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:45:21 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:45:21.340275 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0eb08a4-52fb-4ea6-9c9c-6e9ac7ddea2e service nova] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:21.351498 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0eb08a4-52fb-4ea6-9c9c-6e9ac7ddea2e service nova] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:21.362335 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 810/1622] 10.222.0.81 () {60 vars in 1143 bytes} [Wed Jul 20 22:45:21 2022] POST /identity/v3/users => generated 309 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:45:21.365267 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f10b92-6b4a-4c85-bfa1-903c4f4d7fa4 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:21.375739 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77487fc8-c712-4866-b7ae-652f26d17b9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:21.376573 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77487fc8-c712-4866-b7ae-652f26d17b9a admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:21.376935 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77487fc8-c712-4866-b7ae-652f26d17b9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:21.377285 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77487fc8-c712-4866-b7ae-652f26d17b9a admin admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:21.378309 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77487fc8-c712-4866-b7ae-652f26d17b9a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:21.382168 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77487fc8-c712-4866-b7ae-652f26d17b9a admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:21.382915 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f10b92-6b4a-4c85-bfa1-903c4f4d7fa4 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:21.401444 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172ad47b-ee43-4dff-9c49-90cad31dac71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:21.401444 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172ad47b-ee43-4dff-9c49-90cad31dac71 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:21.401444 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172ad47b-ee43-4dff-9c49-90cad31dac71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:21.401444 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172ad47b-ee43-4dff-9c49-90cad31dac71 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:21.401827 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172ad47b-ee43-4dff-9c49-90cad31dac71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:21.401827 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172ad47b-ee43-4dff-9c49-90cad31dac71 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:21.414349 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 811/1623] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:45:21 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:45:21.420018 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172ad47b-ee43-4dff-9c49-90cad31dac71 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:21.433081 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172ad47b-ee43-4dff-9c49-90cad31dac71 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:21.437969 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bb5247-eb63-4b67-bd06-2a6309d9e26f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:21.441271 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bb5247-eb63-4b67-bd06-2a6309d9e26f admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:21.441271 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bb5247-eb63-4b67-bd06-2a6309d9e26f admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:21.441271 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bb5247-eb63-4b67-bd06-2a6309d9e26f admin admin] PATH_INFO: `/v3/projects/78d0fd5f712a4143ab4b9cce1c9a9a19/users/ede132f5571f4f7cbd6586cf7d9f3c9e/roles/d89b6c92d35a401a8517f1d47f6be843` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:21.456213 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bb5247-eb63-4b67-bd06-2a6309d9e26f admin admin] RBAC: Authorizing `identity:create_grant(project_id=78d0fd5f712a4143ab4b9cce1c9a9a19, user_id=ede132f5571f4f7cbd6586cf7d9f3c9e, role_id=d89b6c92d35a401a8517f1d47f6be843)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:21.459336 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bb5247-eb63-4b67-bd06-2a6309d9e26f admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:21.521902 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 813/1624] 10.222.0.81 () {60 vars in 1143 bytes} [Wed Jul 20 22:45:21 2022] POST /identity/v3/users => generated 300 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:45:21.525643 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77487fc8-c712-4866-b7ae-652f26d17b9a admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:21.536883 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77487fc8-c712-4866-b7ae-652f26d17b9a admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:21.542235 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a94eca-9373-49a6-b968-ab8f036b81a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:21.542870 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a94eca-9373-49a6-b968-ab8f036b81a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:21.543209 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a94eca-9373-49a6-b968-ab8f036b81a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:21.543493 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a94eca-9373-49a6-b968-ab8f036b81a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:21.545201 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a94eca-9373-49a6-b968-ab8f036b81a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:21.545927 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a94eca-9373-49a6-b968-ab8f036b81a2 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:21.560579 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 814/1625] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:45:21 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 22:45:21.573321 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a94eca-9373-49a6-b968-ab8f036b81a2 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:21.594106 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a94eca-9373-49a6-b968-ab8f036b81a2 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:21.600606 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cba0d4-f56e-452c-b3a3-5368e1c9896e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:21.601868 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cba0d4-f56e-452c-b3a3-5368e1c9896e admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:21.602159 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cba0d4-f56e-452c-b3a3-5368e1c9896e admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:21.602542 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cba0d4-f56e-452c-b3a3-5368e1c9896e admin admin] PATH_INFO: `/v3/projects/11914cf18ff643a6a0894f749c5af92c/users/58c8cf41da2e49de90dd4918de0fab6f/roles/d89b6c92d35a401a8517f1d47f6be843` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:21.616891 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 812/1626] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:45:21 2022] PUT /identity/v3/projects/78d0fd5f712a4143ab4b9cce1c9a9a19/users/ede132f5571f4f7cbd6586cf7d9f3c9e/roles/d89b6c92d35a401a8517f1d47f6be843 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:45:21.616891 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bb5247-eb63-4b67-bd06-2a6309d9e26f admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:21.623454 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cba0d4-f56e-452c-b3a3-5368e1c9896e admin admin] RBAC: Authorizing `identity:create_grant(project_id=11914cf18ff643a6a0894f749c5af92c, user_id=58c8cf41da2e49de90dd4918de0fab6f, role_id=d89b6c92d35a401a8517f1d47f6be843)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:21.626937 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cba0d4-f56e-452c-b3a3-5368e1c9896e admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:21.716401 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bb5247-eb63-4b67-bd06-2a6309d9e26f admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:21.719720 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:45:21.722074 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b8882c-6b73-4733-950a-1101dd2c0dc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:21.722658 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b8882c-6b73-4733-950a-1101dd2c0dc0 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:21.723001 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b8882c-6b73-4733-950a-1101dd2c0dc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:21.723337 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b8882c-6b73-4733-950a-1101dd2c0dc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:21.724075 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b8882c-6b73-4733-950a-1101dd2c0dc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:21.725856 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b8882c-6b73-4733-950a-1101dd2c0dc0 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:21.742522 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 813/1627] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:45:21 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 22:45:21.742522 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08709dea-4839-4c90-a782-35c35ca22920 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:21.742522 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08709dea-4839-4c90-a782-35c35ca22920 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:21.742522 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08709dea-4839-4c90-a782-35c35ca22920 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:21.743000 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 814/1628] 10.222.0.81 () {54 vars in 912 bytes} [Wed Jul 20 22:45:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:45:21.747847 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08709dea-4839-4c90-a782-35c35ca22920 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:21.764433 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08709dea-4839-4c90-a782-35c35ca22920 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:21.772448 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 815/1629] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:45:21 2022] PUT /identity/v3/projects/11914cf18ff643a6a0894f749c5af92c/users/58c8cf41da2e49de90dd4918de0fab6f/roles/d89b6c92d35a401a8517f1d47f6be843 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 22:45:21.774175 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f26c53e-97aa-4c22-b21a-717efc187be9 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:21.774508 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f26c53e-97aa-4c22-b21a-717efc187be9 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:21.774797 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f26c53e-97aa-4c22-b21a-717efc187be9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:21.806016 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-0f26c53e-97aa-4c22-b21a-717efc187be9 None None] MFA Rules not processed for user `7829324db2574d38b18c749d0559177f`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:45:21.807932 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-0f26c53e-97aa-4c22-b21a-717efc187be9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:21.834940 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef437ce3-c3f1-43f2-9dd5-ddb49de22dc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:21.835893 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef437ce3-c3f1-43f2-9dd5-ddb49de22dc8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:21.836198 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef437ce3-c3f1-43f2-9dd5-ddb49de22dc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:21.836452 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef437ce3-c3f1-43f2-9dd5-ddb49de22dc8 admin admin] PATH_INFO: `/v3/projects/78d0fd5f712a4143ab4b9cce1c9a9a19/users/ede132f5571f4f7cbd6586cf7d9f3c9e/roles/d89b6c92d35a401a8517f1d47f6be843` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:21.839058 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef437ce3-c3f1-43f2-9dd5-ddb49de22dc8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78d0fd5f712a4143ab4b9cce1c9a9a19, user_id=ede132f5571f4f7cbd6586cf7d9f3c9e, role_id=d89b6c92d35a401a8517f1d47f6be843)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:21.841497 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef437ce3-c3f1-43f2-9dd5-ddb49de22dc8 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:21.853937 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 815/1630] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:45:21 2022] PUT /identity/v3/projects/78d0fd5f712a4143ab4b9cce1c9a9a19/users/ede132f5571f4f7cbd6586cf7d9f3c9e/roles/d89b6c92d35a401a8517f1d47f6be843 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:45:21.855054 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef437ce3-c3f1-43f2-9dd5-ddb49de22dc8 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:21.916780 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 816/1631] 10.222.0.81 () {58 vars in 1009 bytes} [Wed Jul 20 22:45:21 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:45:21.918247 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f92053-48a6-463f-aa89-2a69c80a0f11 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:21.918545 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f92053-48a6-463f-aa89-2a69c80a0f11 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:21.918822 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f92053-48a6-463f-aa89-2a69c80a0f11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:21.954071 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-43f92053-48a6-463f-aa89-2a69c80a0f11 None None] MFA Rules not processed for user `ede132f5571f4f7cbd6586cf7d9f3c9e`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:45:21.961673 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef437ce3-c3f1-43f2-9dd5-ddb49de22dc8 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:21.961815 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-43f92053-48a6-463f-aa89-2a69c80a0f11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:21.961915 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7837d0d-5ceb-4f43-aca2-ba72d1f83b8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:21.963073 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7837d0d-5ceb-4f43-aca2-ba72d1f83b8c admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:21.963451 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7837d0d-5ceb-4f43-aca2-ba72d1f83b8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:21.963807 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7837d0d-5ceb-4f43-aca2-ba72d1f83b8c admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:21.964679 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7837d0d-5ceb-4f43-aca2-ba72d1f83b8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:21.967333 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7837d0d-5ceb-4f43-aca2-ba72d1f83b8c admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:21.986195 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 816/1632] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:45:21 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:45:21.986195 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7837d0d-5ceb-4f43-aca2-ba72d1f83b8c admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:22.053986 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7837d0d-5ceb-4f43-aca2-ba72d1f83b8c admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:22.057724 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 817/1633] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:45:21 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:45:22.061611 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911af671-93c7-40f2-afaa-08128597385d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecf4db289fba4e4a806198c62d31f7c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:22.061611 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911af671-93c7-40f2-afaa-08128597385d service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:22.061611 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911af671-93c7-40f2-afaa-08128597385d service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:22.061611 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911af671-93c7-40f2-afaa-08128597385d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:22.061943 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f92053-48a6-463f-aa89-2a69c80a0f11 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:22.079769 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:45:22.079882 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f92053-48a6-463f-aa89-2a69c80a0f11 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:22.096590 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee811558-d57a-4117-85c3-c05a4794d254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:22.098213 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee811558-d57a-4117-85c3-c05a4794d254 admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:22.099205 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee811558-d57a-4117-85c3-c05a4794d254 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:22.099546 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee811558-d57a-4117-85c3-c05a4794d254 admin admin] PATH_INFO: `/v3/projects/11914cf18ff643a6a0894f749c5af92c/users/58c8cf41da2e49de90dd4918de0fab6f/roles/d89b6c92d35a401a8517f1d47f6be843` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:22.105231 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee811558-d57a-4117-85c3-c05a4794d254 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11914cf18ff643a6a0894f749c5af92c, user_id=58c8cf41da2e49de90dd4918de0fab6f, role_id=d89b6c92d35a401a8517f1d47f6be843)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:22.108745 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee811558-d57a-4117-85c3-c05a4794d254 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:22.125521 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 818/1634] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:45:22 2022] PUT /identity/v3/projects/11914cf18ff643a6a0894f749c5af92c/users/58c8cf41da2e49de90dd4918de0fab6f/roles/d89b6c92d35a401a8517f1d47f6be843 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:45:22.126227 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee811558-d57a-4117-85c3-c05a4794d254 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:22.179923 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911af671-93c7-40f2-afaa-08128597385d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:22.182211 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911af671-93c7-40f2-afaa-08128597385d service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:22.200682 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 817/1635] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:45:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:45:22.202621 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d562c57e-cfe0-499b-ae3c-859f5aaa8f3e None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:22.202621 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d562c57e-cfe0-499b-ae3c-859f5aaa8f3e None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:22.202966 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d562c57e-cfe0-499b-ae3c-859f5aaa8f3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:22.231615 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee811558-d57a-4117-85c3-c05a4794d254 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:22.244919 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c5a269-0eae-4dff-a8ba-a1f7a572aa83 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecf4db289fba4e4a806198c62d31f7c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:22.247208 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-d562c57e-cfe0-499b-ae3c-859f5aaa8f3e None None] MFA Rules not processed for user `58c8cf41da2e49de90dd4918de0fab6f`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:45:22.248058 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c5a269-0eae-4dff-a8ba-a1f7a572aa83 service cinder] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:22.248397 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c5a269-0eae-4dff-a8ba-a1f7a572aa83 service cinder] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:22.248595 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-d562c57e-cfe0-499b-ae3c-859f5aaa8f3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:22.250136 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c5a269-0eae-4dff-a8ba-a1f7a572aa83 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:22.389392 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 818/1636] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:45:22 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:45:22.399217 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c5a269-0eae-4dff-a8ba-a1f7a572aa83 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:22.399761 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fa214f-65e3-4062-a058-023f406bd2e7 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:22.399862 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fa214f-65e3-4062-a058-023f406bd2e7 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:22.400100 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fa214f-65e3-4062-a058-023f406bd2e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:22.404659 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c5a269-0eae-4dff-a8ba-a1f7a572aa83 service cinder] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:22.450657 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 819/1637] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:45:22 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:45:22.465007 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-40fa214f-65e3-4062-a058-023f406bd2e7 None None] MFA Rules not processed for user `3cd5c46e50714155aed3e9f439c2bf6d`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:45:22.470774 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-40fa214f-65e3-4062-a058-023f406bd2e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:22.564489 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c5a269-0eae-4dff-a8ba-a1f7a572aa83 service cinder] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:22.580963 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c5a269-0eae-4dff-a8ba-a1f7a572aa83 service cinder] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:22.593205 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22eb2092-dd24-461a-a748-594bc894d305 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecf4db289fba4e4a806198c62d31f7c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:22.593880 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eb2092-dd24-461a-a748-594bc894d305 service cinder] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:22.594159 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eb2092-dd24-461a-a748-594bc894d305 service cinder] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:22.594503 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22eb2092-dd24-461a-a748-594bc894d305 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:22.600386 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 819/1638] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:45:22 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:45:22.604963 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fa214f-65e3-4062-a058-023f406bd2e7 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:22.631996 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22eb2092-dd24-461a-a748-594bc894d305 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:22.633460 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22eb2092-dd24-461a-a748-594bc894d305 service cinder] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:22.668979 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 820/1639] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:45:22 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 22:45:22.739723 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fa214f-65e3-4062-a058-023f406bd2e7 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:22.746398 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6d0b23-ea1d-4272-82eb-1ce810a9a9ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:22.747448 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6d0b23-ea1d-4272-82eb-1ce810a9a9ee admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:22.747766 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6d0b23-ea1d-4272-82eb-1ce810a9a9ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:22.748144 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6d0b23-ea1d-4272-82eb-1ce810a9a9ee admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:22.749208 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6d0b23-ea1d-4272-82eb-1ce810a9a9ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:22.751323 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6d0b23-ea1d-4272-82eb-1ce810a9a9ee admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:22.825618 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22eb2092-dd24-461a-a748-594bc894d305 service cinder] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:22.843700 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22eb2092-dd24-461a-a748-594bc894d305 service cinder] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:22.857827 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec8853f-f1a8-458a-8735-93144319ad0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:22.858733 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec8853f-f1a8-458a-8735-93144319ad0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:22.859033 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec8853f-f1a8-458a-8735-93144319ad0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:22.859304 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec8853f-f1a8-458a-8735-93144319ad0c admin admin] PATH_INFO: `/v3/users/69b8faa54ac144178b3a00b40ce7bf0d` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:22.864663 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec8853f-f1a8-458a-8735-93144319ad0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=69b8faa54ac144178b3a00b40ce7bf0d)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:22.866913 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec8853f-f1a8-458a-8735-93144319ad0c admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:22.892330 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 820/1640] 10.222.0.81 () {60 vars in 1152 bytes} [Wed Jul 20 22:45:22 2022] POST /identity/v3/projects => generated 405 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 22:45:22.893812 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6d0b23-ea1d-4272-82eb-1ce810a9a9ee admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:22.980896 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6d0b23-ea1d-4272-82eb-1ce810a9a9ee admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:22.987121 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d61993-056a-44dc-92df-3cbf2eff058d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:22.987842 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d61993-056a-44dc-92df-3cbf2eff058d admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:22.988134 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d61993-056a-44dc-92df-3cbf2eff058d admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:22.989045 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d61993-056a-44dc-92df-3cbf2eff058d admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:22.989777 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d61993-056a-44dc-92df-3cbf2eff058d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:22.992238 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d61993-056a-44dc-92df-3cbf2eff058d admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:23.250667 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 821/1641] 10.222.0.81 () {60 vars in 1143 bytes} [Wed Jul 20 22:45:22 2022] POST /identity/v3/users => generated 309 bytes in 357 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:45:23.251649 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d61993-056a-44dc-92df-3cbf2eff058d admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:23.261175 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d61993-056a-44dc-92df-3cbf2eff058d admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:23.266131 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c213c0af-268a-44c5-849b-1af21db83d4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:23.277662 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c213c0af-268a-44c5-849b-1af21db83d4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:23.277662 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c213c0af-268a-44c5-849b-1af21db83d4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:23.277662 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c213c0af-268a-44c5-849b-1af21db83d4f admin admin] PATH_INFO: `/v3/users/3d921323545e4762bd5cb003c5f4852c` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:23.277662 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c213c0af-268a-44c5-849b-1af21db83d4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d921323545e4762bd5cb003c5f4852c)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:23.277662 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: WARNING py.warnings [None req-c213c0af-268a-44c5-849b-1af21db83d4f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 22:45:23.277662 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: warnings.warn(msg) Jul 20 22:45:23.277662 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: Jul 20 22:45:23.278304 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c213c0af-268a-44c5-849b-1af21db83d4f admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:23.348296 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-8ec8853f-f1a8-458a-8735-93144319ad0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69b8faa54ac144178b3a00b40ce7bf0d'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:23.348296 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-8ec8853f-f1a8-458a-8735-93144319ad0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69b8faa54ac144178b3a00b40ce7bf0d'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:23.348296 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-8ec8853f-f1a8-458a-8735-93144319ad0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69b8faa54ac144178b3a00b40ce7bf0d'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:23.359101 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-8ec8853f-f1a8-458a-8735-93144319ad0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69b8faa54ac144178b3a00b40ce7bf0d'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:23.804419 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:45:23.804419 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-8ec8853f-f1a8-458a-8735-93144319ad0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69b8faa54ac144178b3a00b40ce7bf0d'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:23.804419 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 821/1642] 10.222.0.81 () {58 vars in 1223 bytes} [Wed Jul 20 22:45:22 2022] DELETE /identity/v3/users/69b8faa54ac144178b3a00b40ce7bf0d => generated 0 bytes in 858 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 22:45:23.804419 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec8853f-f1a8-458a-8735-93144319ad0c admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:23.804419 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-8ec8853f-f1a8-458a-8735-93144319ad0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:23.804419 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec8853f-f1a8-458a-8735-93144319ad0c admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:23.806817 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cef52f7-09aa-4806-a4c2-8badc4e7eb4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:23.806817 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cef52f7-09aa-4806-a4c2-8badc4e7eb4d admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:23.806817 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cef52f7-09aa-4806-a4c2-8badc4e7eb4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:23.806817 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cef52f7-09aa-4806-a4c2-8badc4e7eb4d admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:23.807504 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cef52f7-09aa-4806-a4c2-8badc4e7eb4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:23.807504 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cef52f7-09aa-4806-a4c2-8badc4e7eb4d admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:23.807504 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 822/1643] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:45:23 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:45:23.807504 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b23b80-db29-45b0-a169-4781d9e1b45e None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:23.807504 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b23b80-db29-45b0-a169-4781d9e1b45e None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:23.807504 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b23b80-db29-45b0-a169-4781d9e1b45e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:23.808406 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-64b23b80-db29-45b0-a169-4781d9e1b45e None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:45:23.808406 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-64b23b80-db29-45b0-a169-4781d9e1b45e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:23.834759 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 823/1644] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:45:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:45:23.834759 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b23b80-db29-45b0-a169-4781d9e1b45e None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:23.834759 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b23b80-db29-45b0-a169-4781d9e1b45e None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:23.844108 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb4ee92-b23f-42ba-98ef-c6435394576c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:23.845843 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb4ee92-b23f-42ba-98ef-c6435394576c admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:23.846041 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb4ee92-b23f-42ba-98ef-c6435394576c admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:23.846442 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb4ee92-b23f-42ba-98ef-c6435394576c admin admin] PATH_INFO: `/v3/projects/39aa4cce7199466fbd878465214e4137/users/fa9199f34c1048ed8e756b4fa324e606/roles/d89b6c92d35a401a8517f1d47f6be843` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:23.862800 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb4ee92-b23f-42ba-98ef-c6435394576c admin admin] RBAC: Authorizing `identity:create_grant(project_id=39aa4cce7199466fbd878465214e4137, user_id=fa9199f34c1048ed8e756b4fa324e606, role_id=d89b6c92d35a401a8517f1d47f6be843)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:23.865605 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb4ee92-b23f-42ba-98ef-c6435394576c admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:23.880821 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-c213c0af-268a-44c5-849b-1af21db83d4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d921323545e4762bd5cb003c5f4852c'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:23.881482 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-c213c0af-268a-44c5-849b-1af21db83d4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d921323545e4762bd5cb003c5f4852c'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:23.882121 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-c213c0af-268a-44c5-849b-1af21db83d4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d921323545e4762bd5cb003c5f4852c'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:23.893165 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-c213c0af-268a-44c5-849b-1af21db83d4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d921323545e4762bd5cb003c5f4852c'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:24.065950 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-c213c0af-268a-44c5-849b-1af21db83d4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d921323545e4762bd5cb003c5f4852c'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:24.085979 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 824/1645] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:45:23 2022] PUT /identity/v3/projects/39aa4cce7199466fbd878465214e4137/users/fa9199f34c1048ed8e756b4fa324e606/roles/d89b6c92d35a401a8517f1d47f6be843 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:45:24.087517 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb4ee92-b23f-42ba-98ef-c6435394576c admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:24.091852 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-ceb4ee92-b23f-42ba-98ef-c6435394576c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:24.139567 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 822/1646] 10.222.0.81 () {58 vars in 1223 bytes} [Wed Jul 20 22:45:23 2022] DELETE /identity/v3/users/3d921323545e4762bd5cb003c5f4852c => generated 0 bytes in 885 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:45:24.140549 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c213c0af-268a-44c5-849b-1af21db83d4f admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:24.144001 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-c213c0af-268a-44c5-849b-1af21db83d4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:24.247730 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb4ee92-b23f-42ba-98ef-c6435394576c admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:24.258659 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:45:24.311356 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c213c0af-268a-44c5-849b-1af21db83d4f admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:24.329218 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a271cb-7359-4be6-b3d3-7def7b5fe3ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:24.329987 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a271cb-7359-4be6-b3d3-7def7b5fe3ae admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:24.330273 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a271cb-7359-4be6-b3d3-7def7b5fe3ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:24.330635 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a271cb-7359-4be6-b3d3-7def7b5fe3ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:24.331331 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a271cb-7359-4be6-b3d3-7def7b5fe3ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:24.332957 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a271cb-7359-4be6-b3d3-7def7b5fe3ae admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:24.357134 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 823/1647] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:45:24 2022] GET /identity/v3/roles => generated 1420 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:45:24.362504 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915f0a57-a2ae-492a-9d49-c00af50c9875 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:24.362809 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915f0a57-a2ae-492a-9d49-c00af50c9875 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:24.363114 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915f0a57-a2ae-492a-9d49-c00af50c9875 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:24.372646 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d1b657-12a0-4283-a4e6-fe5ba9a5de8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:24.373363 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d1b657-12a0-4283-a4e6-fe5ba9a5de8b service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:24.373650 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d1b657-12a0-4283-a4e6-fe5ba9a5de8b service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:24.373969 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d1b657-12a0-4283-a4e6-fe5ba9a5de8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:24.380414 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-37d1b657-12a0-4283-a4e6-fe5ba9a5de8b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:24.402262 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-915f0a57-a2ae-492a-9d49-c00af50c9875 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:45:24.405082 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-915f0a57-a2ae-492a-9d49-c00af50c9875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:24.408027 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d1b657-12a0-4283-a4e6-fe5ba9a5de8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:24.409604 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d1b657-12a0-4283-a4e6-fe5ba9a5de8b service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:24.436312 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 824/1648] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:45:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:45:24.438042 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915f0a57-a2ae-492a-9d49-c00af50c9875 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:24.446766 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 825/1649] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:45:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:45:24.452047 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915f0a57-a2ae-492a-9d49-c00af50c9875 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:24.458342 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a81436-d7e7-468a-96f5-8f190cad10c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:24.459347 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a81436-d7e7-468a-96f5-8f190cad10c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:24.459679 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a81436-d7e7-468a-96f5-8f190cad10c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:24.460177 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a81436-d7e7-468a-96f5-8f190cad10c4 admin admin] PATH_INFO: `/v3/projects/39aa4cce7199466fbd878465214e4137/users/fa9199f34c1048ed8e756b4fa324e606/roles/d89b6c92d35a401a8517f1d47f6be843` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:24.462087 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a81436-d7e7-468a-96f5-8f190cad10c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=39aa4cce7199466fbd878465214e4137, user_id=fa9199f34c1048ed8e756b4fa324e606, role_id=d89b6c92d35a401a8517f1d47f6be843)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:24.464839 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a81436-d7e7-468a-96f5-8f190cad10c4 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:24.479096 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 825/1650] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:45:24 2022] PUT /identity/v3/projects/39aa4cce7199466fbd878465214e4137/users/fa9199f34c1048ed8e756b4fa324e606/roles/d89b6c92d35a401a8517f1d47f6be843 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:45:24.489004 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d1b657-12a0-4283-a4e6-fe5ba9a5de8b service neutron] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:24.499638 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d1b657-12a0-4283-a4e6-fe5ba9a5de8b service neutron] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:24.591884 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb6c41e-9c04-4869-8085-464a32562e42 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:24.592478 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb6c41e-9c04-4869-8085-464a32562e42 service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:24.592815 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb6c41e-9c04-4869-8085-464a32562e42 service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:24.593262 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb6c41e-9c04-4869-8085-464a32562e42 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:24.654972 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70da0a67-a28a-4db3-bc42-65224519120f None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:24.655495 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70da0a67-a28a-4db3-bc42-65224519120f None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:24.656066 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70da0a67-a28a-4db3-bc42-65224519120f None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:24.657645 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 826/1651] 10.222.0.81 () {54 vars in 962 bytes} [Wed Jul 20 22:45:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 22:45:24.669628 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bb54ec-a4ae-4c84-b2b0-824f2a46c59e None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:24.670123 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bb54ec-a4ae-4c84-b2b0-824f2a46c59e None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:24.670538 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bb54ec-a4ae-4c84-b2b0-824f2a46c59e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:24.712189 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-17bb54ec-a4ae-4c84-b2b0-824f2a46c59e None None] MFA Rules not processed for user `7829324db2574d38b18c749d0559177f`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:45:24.713452 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-17bb54ec-a4ae-4c84-b2b0-824f2a46c59e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:24.724144 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb6c41e-9c04-4869-8085-464a32562e42 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:24.725784 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb6c41e-9c04-4869-8085-464a32562e42 service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:24.742805 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 826/1652] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:45:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 22:45:24.803178 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 827/1653] 10.222.0.81 () {58 vars in 1059 bytes} [Wed Jul 20 22:45:24 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 22:45:24.808998 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8303e316-1a8c-4393-b800-991a9914f570 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:24.809508 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8303e316-1a8c-4393-b800-991a9914f570 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:24.809626 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8303e316-1a8c-4393-b800-991a9914f570 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:24.811162 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 827/1654] 10.222.0.81 () {54 vars in 912 bytes} [Wed Jul 20 22:45:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 22:45:24.816536 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca4677d-dcdd-456c-b06a-e0d35f56868c None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:24.816834 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca4677d-dcdd-456c-b06a-e0d35f56868c None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:24.817285 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca4677d-dcdd-456c-b06a-e0d35f56868c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:24.843628 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-eca4677d-dcdd-456c-b06a-e0d35f56868c None None] MFA Rules not processed for user `7829324db2574d38b18c749d0559177f`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:45:24.845708 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-eca4677d-dcdd-456c-b06a-e0d35f56868c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:24.865130 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 828/1655] 10.222.0.81 () {58 vars in 1009 bytes} [Wed Jul 20 22:45:24 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:45:24.872976 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8303e316-1a8c-4393-b800-991a9914f570 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:24.875974 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-8303e316-1a8c-4393-b800-991a9914f570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:24.968905 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8303e316-1a8c-4393-b800-991a9914f570 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:24.974867 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e66742-3ec2-4327-88b0-d6d30ba27cf4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecf4db289fba4e4a806198c62d31f7c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:24.975603 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e66742-3ec2-4327-88b0-d6d30ba27cf4 service cinder] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:24.976037 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e66742-3ec2-4327-88b0-d6d30ba27cf4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:24.976380 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e66742-3ec2-4327-88b0-d6d30ba27cf4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:24.991075 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e66742-3ec2-4327-88b0-d6d30ba27cf4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:24.991763 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e66742-3ec2-4327-88b0-d6d30ba27cf4 service cinder] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:25.033643 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 828/1656] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:45:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:45:25.353168 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:45:25.862339 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:45:26.366117 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca4677d-dcdd-456c-b06a-e0d35f56868c None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:26.368401 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-eca4677d-dcdd-456c-b06a-e0d35f56868c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:26.445730 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca4677d-dcdd-456c-b06a-e0d35f56868c None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:26.463755 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715761dd-eaf2-4260-81c8-3e28033385db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:26.472159 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715761dd-eaf2-4260-81c8-3e28033385db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:26.474897 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715761dd-eaf2-4260-81c8-3e28033385db admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:26.475543 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715761dd-eaf2-4260-81c8-3e28033385db admin admin] PATH_INFO: `/v3/projects/a3f935addf9a4578871c37310f0eb093` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:26.480386 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:45:26.481771 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715761dd-eaf2-4260-81c8-3e28033385db admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3f935addf9a4578871c37310f0eb093)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:26.485058 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715761dd-eaf2-4260-81c8-3e28033385db admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:26.673965 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-715761dd-eaf2-4260-81c8-3e28033385db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3f935addf9a4578871c37310f0eb093', 'request_id': 'req-715761dd-eaf2-4260-81c8-3e28033385db'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:26.685196 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-715761dd-eaf2-4260-81c8-3e28033385db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3f935addf9a4578871c37310f0eb093', 'request_id': 'req-715761dd-eaf2-4260-81c8-3e28033385db'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:26.775823 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-715761dd-eaf2-4260-81c8-3e28033385db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3f935addf9a4578871c37310f0eb093', 'request_id': 'req-715761dd-eaf2-4260-81c8-3e28033385db'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:26.798319 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-715761dd-eaf2-4260-81c8-3e28033385db admin admin] The token cache is being invalidate because project a3f935addf9a4578871c37310f0eb093 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:45:26.803272 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-715761dd-eaf2-4260-81c8-3e28033385db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:26.805226 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-715761dd-eaf2-4260-81c8-3e28033385db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:26.815908 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 829/1657] 10.222.0.81 () {58 vars in 1232 bytes} [Wed Jul 20 22:45:26 2022] DELETE /identity/v3/projects/a3f935addf9a4578871c37310f0eb093 => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:45:26.821477 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e66742-3ec2-4327-88b0-d6d30ba27cf4 service cinder] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:26.843265 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e66742-3ec2-4327-88b0-d6d30ba27cf4 service cinder] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:26.876293 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:45:26.963416 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e180061-c8ff-48fb-a6ad-49b07ec98b27 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:26.963416 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e180061-c8ff-48fb-a6ad-49b07ec98b27 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:26.963416 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e180061-c8ff-48fb-a6ad-49b07ec98b27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:27.020178 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d18dabb-11e1-4367-8102-89be30a07ee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:27.021072 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d18dabb-11e1-4367-8102-89be30a07ee5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:27.021507 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d18dabb-11e1-4367-8102-89be30a07ee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:27.028756 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d18dabb-11e1-4367-8102-89be30a07ee5 admin admin] PATH_INFO: `/v3/users/ede132f5571f4f7cbd6586cf7d9f3c9e` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:27.028756 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d18dabb-11e1-4367-8102-89be30a07ee5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ede132f5571f4f7cbd6586cf7d9f3c9e)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:27.032136 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d18dabb-11e1-4367-8102-89be30a07ee5 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:27.083067 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-9e180061-c8ff-48fb-a6ad-49b07ec98b27 None None] MFA Rules not processed for user `fa9199f34c1048ed8e756b4fa324e606`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:45:27.084422 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-9e180061-c8ff-48fb-a6ad-49b07ec98b27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:27.277141 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 830/1658] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:45:26 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:45:27.387983 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-6d18dabb-11e1-4367-8102-89be30a07ee5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ede132f5571f4f7cbd6586cf7d9f3c9e'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:27.391240 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-6d18dabb-11e1-4367-8102-89be30a07ee5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ede132f5571f4f7cbd6586cf7d9f3c9e'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:27.392202 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-6d18dabb-11e1-4367-8102-89be30a07ee5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ede132f5571f4f7cbd6586cf7d9f3c9e'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:27.414619 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-6d18dabb-11e1-4367-8102-89be30a07ee5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ede132f5571f4f7cbd6586cf7d9f3c9e'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:27.541717 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e180061-c8ff-48fb-a6ad-49b07ec98b27 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:27.542293 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-6d18dabb-11e1-4367-8102-89be30a07ee5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ede132f5571f4f7cbd6586cf7d9f3c9e'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:27.560834 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-9e180061-c8ff-48fb-a6ad-49b07ec98b27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:27.575247 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 829/1659] 10.222.0.81 () {58 vars in 1223 bytes} [Wed Jul 20 22:45:26 2022] DELETE /identity/v3/users/ede132f5571f4f7cbd6586cf7d9f3c9e => generated 0 bytes in 755 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:45:27.602285 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e180061-c8ff-48fb-a6ad-49b07ec98b27 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:27.714804 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80746492-ff3f-4401-95cd-7cadeb0a8ab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:27.715948 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80746492-ff3f-4401-95cd-7cadeb0a8ab7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:27.716386 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80746492-ff3f-4401-95cd-7cadeb0a8ab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:27.716808 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80746492-ff3f-4401-95cd-7cadeb0a8ab7 admin admin] PATH_INFO: `/v3/projects/7c71439943d84ea888b6b20f2d78a0cf` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:27.736290 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80746492-ff3f-4401-95cd-7cadeb0a8ab7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c71439943d84ea888b6b20f2d78a0cf)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:27.769691 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80746492-ff3f-4401-95cd-7cadeb0a8ab7 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:27.943287 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-80746492-ff3f-4401-95cd-7cadeb0a8ab7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c71439943d84ea888b6b20f2d78a0cf', 'request_id': 'req-80746492-ff3f-4401-95cd-7cadeb0a8ab7'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:27.960753 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-80746492-ff3f-4401-95cd-7cadeb0a8ab7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c71439943d84ea888b6b20f2d78a0cf', 'request_id': 'req-80746492-ff3f-4401-95cd-7cadeb0a8ab7'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:28.062154 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-80746492-ff3f-4401-95cd-7cadeb0a8ab7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c71439943d84ea888b6b20f2d78a0cf', 'request_id': 'req-80746492-ff3f-4401-95cd-7cadeb0a8ab7'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:28.089394 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-80746492-ff3f-4401-95cd-7cadeb0a8ab7 admin admin] The token cache is being invalidate because project 7c71439943d84ea888b6b20f2d78a0cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:45:28.101468 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-80746492-ff3f-4401-95cd-7cadeb0a8ab7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:28.105640 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-80746492-ff3f-4401-95cd-7cadeb0a8ab7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:28.121417 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 831/1660] 10.222.0.81 () {58 vars in 1232 bytes} [Wed Jul 20 22:45:27 2022] DELETE /identity/v3/projects/7c71439943d84ea888b6b20f2d78a0cf => generated 0 bytes in 577 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:45:28.121910 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d18dabb-11e1-4367-8102-89be30a07ee5 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:28.123414 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-6d18dabb-11e1-4367-8102-89be30a07ee5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:28.152909 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d18dabb-11e1-4367-8102-89be30a07ee5 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:28.330460 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e020993-fad5-4f73-963c-4fd99cd938c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:28.330460 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e020993-fad5-4f73-963c-4fd99cd938c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:28.330460 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e020993-fad5-4f73-963c-4fd99cd938c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:28.330460 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e020993-fad5-4f73-963c-4fd99cd938c8 admin admin] PATH_INFO: `/v3/users/58c8cf41da2e49de90dd4918de0fab6f` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:28.331745 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e020993-fad5-4f73-963c-4fd99cd938c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=58c8cf41da2e49de90dd4918de0fab6f)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:28.333806 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e020993-fad5-4f73-963c-4fd99cd938c8 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:28.400409 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:45:28.592786 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-2e020993-fad5-4f73-963c-4fd99cd938c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58c8cf41da2e49de90dd4918de0fab6f'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:28.593920 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-2e020993-fad5-4f73-963c-4fd99cd938c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58c8cf41da2e49de90dd4918de0fab6f'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:28.594752 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-2e020993-fad5-4f73-963c-4fd99cd938c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58c8cf41da2e49de90dd4918de0fab6f'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:28.604847 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-2e020993-fad5-4f73-963c-4fd99cd938c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58c8cf41da2e49de90dd4918de0fab6f'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:28.687650 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-2e020993-fad5-4f73-963c-4fd99cd938c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58c8cf41da2e49de90dd4918de0fab6f'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:28.728723 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 830/1661] 10.222.0.81 () {58 vars in 1223 bytes} [Wed Jul 20 22:45:28 2022] DELETE /identity/v3/users/58c8cf41da2e49de90dd4918de0fab6f => generated 0 bytes in 607 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 22:45:28.754516 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:45:29.109628 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca74e78-d5b7-4d18-bf84-1aa9ee311892 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:29.109628 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca74e78-d5b7-4d18-bf84-1aa9ee311892 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:29.109628 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca74e78-d5b7-4d18-bf84-1aa9ee311892 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:29.112237 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 832/1662] 10.222.0.81 () {54 vars in 921 bytes} [Wed Jul 20 22:45:29 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:45:29.161565 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dfd45f-6d26-4ef7-83b9-1c513e0177f5 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:29.162068 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dfd45f-6d26-4ef7-83b9-1c513e0177f5 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:29.171573 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dfd45f-6d26-4ef7-83b9-1c513e0177f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:29.233741 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-33dfd45f-6d26-4ef7-83b9-1c513e0177f5 None None] MFA Rules not processed for user `d5500cebdd9d4f8e9aecb45d6c955878`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:45:29.236056 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-33dfd45f-6d26-4ef7-83b9-1c513e0177f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:29.383613 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 831/1663] 10.222.0.81 () {58 vars in 1018 bytes} [Wed Jul 20 22:45:29 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:45:29.410113 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859007aa-0124-4d42-8ea8-a4aab3fb5f1d None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:29.411336 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859007aa-0124-4d42-8ea8-a4aab3fb5f1d None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:29.411850 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859007aa-0124-4d42-8ea8-a4aab3fb5f1d None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:29.416215 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 833/1664] 10.222.0.81 () {54 vars in 961 bytes} [Wed Jul 20 22:45:29 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:45:29.424943 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3d8511-6770-4be3-86b1-f17833cca092 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:29.425240 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3d8511-6770-4be3-86b1-f17833cca092 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:29.425730 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3d8511-6770-4be3-86b1-f17833cca092 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:29.458393 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-df3d8511-6770-4be3-86b1-f17833cca092 None None] MFA Rules not processed for user `d5500cebdd9d4f8e9aecb45d6c955878`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:45:29.459764 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-df3d8511-6770-4be3-86b1-f17833cca092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:29.514276 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 832/1665] 10.222.0.81 () {58 vars in 1058 bytes} [Wed Jul 20 22:45:29 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:45:29.515658 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859007aa-0124-4d42-8ea8-a4aab3fb5f1d None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:29.568466 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859007aa-0124-4d42-8ea8-a4aab3fb5f1d None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:29.584934 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6964608-8443-43d4-b66a-3318bc801738 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5500cebdd9d4f8e9aecb45d6c955878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:29.587094 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6964608-8443-43d4-b66a-3318bc801738 service placement] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:29.588668 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6964608-8443-43d4-b66a-3318bc801738 service placement] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:29.590100 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6964608-8443-43d4-b66a-3318bc801738 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:29.593428 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:45:29.606550 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6964608-8443-43d4-b66a-3318bc801738 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:29.610284 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6964608-8443-43d4-b66a-3318bc801738 service placement] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:29.622508 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 834/1666] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:45:29 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:45:29.693849 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3d8511-6770-4be3-86b1-f17833cca092 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:29.700057 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-df3d8511-6770-4be3-86b1-f17833cca092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:29.837883 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3d8511-6770-4be3-86b1-f17833cca092 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:29.848744 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d5880a-0b83-4fc1-8fa0-ee7c028eb448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:29.851897 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d5880a-0b83-4fc1-8fa0-ee7c028eb448 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:29.852192 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d5880a-0b83-4fc1-8fa0-ee7c028eb448 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:29.852473 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d5880a-0b83-4fc1-8fa0-ee7c028eb448 admin admin] PATH_INFO: `/v3/projects/78d0fd5f712a4143ab4b9cce1c9a9a19` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:29.855741 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d5880a-0b83-4fc1-8fa0-ee7c028eb448 admin admin] RBAC: Authorizing `identity:delete_project(project_id=78d0fd5f712a4143ab4b9cce1c9a9a19)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:29.856828 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: WARNING py.warnings [None req-d3d5880a-0b83-4fc1-8fa0-ee7c028eb448 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 22:45:29.856828 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: warnings.warn(msg) Jul 20 22:45:29.856828 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: Jul 20 22:45:29.858043 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d5880a-0b83-4fc1-8fa0-ee7c028eb448 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:30.067464 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-d3d5880a-0b83-4fc1-8fa0-ee7c028eb448 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78d0fd5f712a4143ab4b9cce1c9a9a19', 'request_id': 'req-d3d5880a-0b83-4fc1-8fa0-ee7c028eb448'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:30.194693 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-d3d5880a-0b83-4fc1-8fa0-ee7c028eb448 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78d0fd5f712a4143ab4b9cce1c9a9a19', 'request_id': 'req-d3d5880a-0b83-4fc1-8fa0-ee7c028eb448'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:30.204657 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-d3d5880a-0b83-4fc1-8fa0-ee7c028eb448 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78d0fd5f712a4143ab4b9cce1c9a9a19', 'request_id': 'req-d3d5880a-0b83-4fc1-8fa0-ee7c028eb448'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:30.261495 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-d3d5880a-0b83-4fc1-8fa0-ee7c028eb448 admin admin] The token cache is being invalidate because project 78d0fd5f712a4143ab4b9cce1c9a9a19 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:45:30.262392 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-d3d5880a-0b83-4fc1-8fa0-ee7c028eb448 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:30.264798 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-d3d5880a-0b83-4fc1-8fa0-ee7c028eb448 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:30.275411 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 833/1667] 10.222.0.81 () {58 vars in 1232 bytes} [Wed Jul 20 22:45:29 2022] DELETE /identity/v3/projects/78d0fd5f712a4143ab4b9cce1c9a9a19 => generated 0 bytes in 577 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:45:30.278227 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e8148d-e088-406d-b09a-59e42fc75d8a None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:30.278227 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e8148d-e088-406d-b09a-59e42fc75d8a None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:30.278227 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e8148d-e088-406d-b09a-59e42fc75d8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:30.278921 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d5880a-0b83-4fc1-8fa0-ee7c028eb448 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:30.286291 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-d3d5880a-0b83-4fc1-8fa0-ee7c028eb448 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:30.328472 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-83e8148d-e088-406d-b09a-59e42fc75d8a None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:45:30.331723 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-83e8148d-e088-406d-b09a-59e42fc75d8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:30.351618 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:45:30.444064 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 835/1668] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:45:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:45:30.456728 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45773117-dc9a-47bf-a2ea-5ea11f2ca67a None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:30.460676 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45773117-dc9a-47bf-a2ea-5ea11f2ca67a None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:30.462369 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d5880a-0b83-4fc1-8fa0-ee7c028eb448 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:30.462831 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45773117-dc9a-47bf-a2ea-5ea11f2ca67a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:30.476908 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0e5865-9789-4a0f-afdd-bce1fbf0a2a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:30.477818 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0e5865-9789-4a0f-afdd-bce1fbf0a2a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:30.478103 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0e5865-9789-4a0f-afdd-bce1fbf0a2a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:30.478381 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0e5865-9789-4a0f-afdd-bce1fbf0a2a8 admin admin] PATH_INFO: `/v3/projects/11914cf18ff643a6a0894f749c5af92c` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:30.484126 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0e5865-9789-4a0f-afdd-bce1fbf0a2a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=11914cf18ff643a6a0894f749c5af92c)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:30.486153 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0e5865-9789-4a0f-afdd-bce1fbf0a2a8 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:30.505996 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-45773117-dc9a-47bf-a2ea-5ea11f2ca67a None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:45:30.507453 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-45773117-dc9a-47bf-a2ea-5ea11f2ca67a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:30.530751 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 836/1669] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:45:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:45:30.536116 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45773117-dc9a-47bf-a2ea-5ea11f2ca67a None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:30.550090 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45773117-dc9a-47bf-a2ea-5ea11f2ca67a None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:30.555528 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30a886e-047c-4cf1-af0f-736e7726b1e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:30.556770 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30a886e-047c-4cf1-af0f-736e7726b1e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:30.557288 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30a886e-047c-4cf1-af0f-736e7726b1e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:30.557742 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30a886e-047c-4cf1-af0f-736e7726b1e2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:30.559300 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30a886e-047c-4cf1-af0f-736e7726b1e2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:45:30.559728 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30a886e-047c-4cf1-af0f-736e7726b1e2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:30.561744 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30a886e-047c-4cf1-af0f-736e7726b1e2 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:30.583130 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 837/1670] 10.222.0.81 () {58 vars in 1152 bytes} [Wed Jul 20 22:45:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 22:45:30.585151 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30a886e-047c-4cf1-af0f-736e7726b1e2 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:30.601443 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30a886e-047c-4cf1-af0f-736e7726b1e2 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:30.614044 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdafeab-6596-4d35-aee6-44fecbda9b53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:30.615081 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdafeab-6596-4d35-aee6-44fecbda9b53 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:30.616084 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdafeab-6596-4d35-aee6-44fecbda9b53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:30.617766 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdafeab-6596-4d35-aee6-44fecbda9b53 admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:30.617766 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdafeab-6596-4d35-aee6-44fecbda9b53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:30.624078 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdafeab-6596-4d35-aee6-44fecbda9b53 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:30.645120 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-af0e5865-9789-4a0f-afdd-bce1fbf0a2a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11914cf18ff643a6a0894f749c5af92c', 'request_id': 'req-af0e5865-9789-4a0f-afdd-bce1fbf0a2a8'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:30.706914 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-af0e5865-9789-4a0f-afdd-bce1fbf0a2a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11914cf18ff643a6a0894f749c5af92c', 'request_id': 'req-af0e5865-9789-4a0f-afdd-bce1fbf0a2a8'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:30.715193 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-af0e5865-9789-4a0f-afdd-bce1fbf0a2a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11914cf18ff643a6a0894f749c5af92c', 'request_id': 'req-af0e5865-9789-4a0f-afdd-bce1fbf0a2a8'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:30.733955 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-af0e5865-9789-4a0f-afdd-bce1fbf0a2a8 admin admin] The token cache is being invalidate because project 11914cf18ff643a6a0894f749c5af92c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:45:30.734323 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-af0e5865-9789-4a0f-afdd-bce1fbf0a2a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:30.735511 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-af0e5865-9789-4a0f-afdd-bce1fbf0a2a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:30.737711 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 834/1671] 10.222.0.81 () {58 vars in 1232 bytes} [Wed Jul 20 22:45:30 2022] DELETE /identity/v3/projects/11914cf18ff643a6a0894f749c5af92c => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:45:30.742395 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0799c4-7b7d-4c02-a0de-da635c1ced75 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:30.743086 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0799c4-7b7d-4c02-a0de-da635c1ced75 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:30.743778 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0799c4-7b7d-4c02-a0de-da635c1ced75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:30.765880 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-7f0799c4-7b7d-4c02-a0de-da635c1ced75 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:45:30.767870 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-7f0799c4-7b7d-4c02-a0de-da635c1ced75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:30.809461 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 838/1672] 10.222.0.81 () {60 vars in 1152 bytes} [Wed Jul 20 22:45:30 2022] POST /identity/v3/projects => generated 371 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 22:45:30.814963 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdafeab-6596-4d35-aee6-44fecbda9b53 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:30.817228 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-fcdafeab-6596-4d35-aee6-44fecbda9b53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:30.841836 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 835/1673] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:45:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:45:30.851450 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327d26c8-bfd4-4616-83fc-84a124739177 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:30.851611 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327d26c8-bfd4-4616-83fc-84a124739177 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:30.852085 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327d26c8-bfd4-4616-83fc-84a124739177 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:30.874776 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-327d26c8-bfd4-4616-83fc-84a124739177 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:45:30.876804 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-327d26c8-bfd4-4616-83fc-84a124739177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:30.909386 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdafeab-6596-4d35-aee6-44fecbda9b53 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:30.919600 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f9527b-8baa-4556-9d13-a1ad33bbf48a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:30.921980 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f9527b-8baa-4556-9d13-a1ad33bbf48a admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:30.922483 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f9527b-8baa-4556-9d13-a1ad33bbf48a admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:30.923141 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f9527b-8baa-4556-9d13-a1ad33bbf48a admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:30.924772 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f9527b-8baa-4556-9d13-a1ad33bbf48a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:30.929157 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f9527b-8baa-4556-9d13-a1ad33bbf48a admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:30.963996 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 836/1674] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:45:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:45:30.969502 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327d26c8-bfd4-4616-83fc-84a124739177 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:30.993452 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327d26c8-bfd4-4616-83fc-84a124739177 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:30.996876 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83513626-82d1-442e-8cb2-320e66de3a58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:30.997817 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83513626-82d1-442e-8cb2-320e66de3a58 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:30.998728 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83513626-82d1-442e-8cb2-320e66de3a58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:30.998728 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83513626-82d1-442e-8cb2-320e66de3a58 admin admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:30.999657 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83513626-82d1-442e-8cb2-320e66de3a58 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:45:31.001841 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83513626-82d1-442e-8cb2-320e66de3a58 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:31.002507 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83513626-82d1-442e-8cb2-320e66de3a58 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:31.019380 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 837/1675] 10.222.0.81 () {58 vars in 1152 bytes} [Wed Jul 20 22:45:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 22:45:31.023392 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83513626-82d1-442e-8cb2-320e66de3a58 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:31.036929 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83513626-82d1-442e-8cb2-320e66de3a58 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:31.045030 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36848a3-571a-4e4e-997b-11f42a36e4fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:31.045907 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36848a3-571a-4e4e-997b-11f42a36e4fc admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:31.046318 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36848a3-571a-4e4e-997b-11f42a36e4fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:31.047210 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36848a3-571a-4e4e-997b-11f42a36e4fc admin admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:31.050847 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36848a3-571a-4e4e-997b-11f42a36e4fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:31.053080 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36848a3-571a-4e4e-997b-11f42a36e4fc admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:31.062646 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 839/1676] 10.222.0.81 () {60 vars in 1143 bytes} [Wed Jul 20 22:45:30 2022] POST /identity/v3/users => generated 286 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:45:31.066384 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f9527b-8baa-4556-9d13-a1ad33bbf48a admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:31.076984 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f9527b-8baa-4556-9d13-a1ad33bbf48a admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:31.083639 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6749a239-7e5f-4206-bdfc-a0ac7677d045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:31.084458 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6749a239-7e5f-4206-bdfc-a0ac7677d045 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:31.085261 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6749a239-7e5f-4206-bdfc-a0ac7677d045 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:31.085982 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6749a239-7e5f-4206-bdfc-a0ac7677d045 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:31.087049 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6749a239-7e5f-4206-bdfc-a0ac7677d045 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:31.088465 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6749a239-7e5f-4206-bdfc-a0ac7677d045 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:31.099801 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 840/1677] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:45:31 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:45:31.103604 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6749a239-7e5f-4206-bdfc-a0ac7677d045 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:31.113571 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6749a239-7e5f-4206-bdfc-a0ac7677d045 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:31.134350 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3331a6-a2a1-4171-ae03-88e743b97bc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:31.134350 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3331a6-a2a1-4171-ae03-88e743b97bc4 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:31.134350 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3331a6-a2a1-4171-ae03-88e743b97bc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:31.134350 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3331a6-a2a1-4171-ae03-88e743b97bc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:31.134618 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3331a6-a2a1-4171-ae03-88e743b97bc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:31.134618 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3331a6-a2a1-4171-ae03-88e743b97bc4 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:31.145898 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 841/1678] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:45:31 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 22:45:31.147973 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3331a6-a2a1-4171-ae03-88e743b97bc4 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:31.163771 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 838/1679] 10.222.0.81 () {60 vars in 1152 bytes} [Wed Jul 20 22:45:31 2022] POST /identity/v3/projects => generated 387 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 22:45:31.169005 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3331a6-a2a1-4171-ae03-88e743b97bc4 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:31.175399 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36848a3-571a-4e4e-997b-11f42a36e4fc admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:31.254177 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bc2b74-5c2a-41e2-8895-d0e8bcf57203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:31.255676 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bc2b74-5c2a-41e2-8895-d0e8bcf57203 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:31.256071 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bc2b74-5c2a-41e2-8895-d0e8bcf57203 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:31.256762 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bc2b74-5c2a-41e2-8895-d0e8bcf57203 admin admin] PATH_INFO: `/v3/projects/c66dad1ba27644a5ae96c9c6080f46d6/users/d43fe9125f6243a796d0d98ef1456182/roles/559942c4de1744169f8b5e0404ecde6b` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:31.276532 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bc2b74-5c2a-41e2-8895-d0e8bcf57203 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c66dad1ba27644a5ae96c9c6080f46d6, user_id=d43fe9125f6243a796d0d98ef1456182, role_id=559942c4de1744169f8b5e0404ecde6b)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:31.279607 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bc2b74-5c2a-41e2-8895-d0e8bcf57203 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:31.285690 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36848a3-571a-4e4e-997b-11f42a36e4fc admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:31.290476 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7a920a-55b8-49a3-9361-4a55761022eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:31.291269 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7a920a-55b8-49a3-9361-4a55761022eb admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:31.292174 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7a920a-55b8-49a3-9361-4a55761022eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:31.292174 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7a920a-55b8-49a3-9361-4a55761022eb admin admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:31.292585 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7a920a-55b8-49a3-9361-4a55761022eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:31.294581 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7a920a-55b8-49a3-9361-4a55761022eb admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:31.393442 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 842/1680] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:45:31 2022] PUT /identity/v3/projects/c66dad1ba27644a5ae96c9c6080f46d6/users/d43fe9125f6243a796d0d98ef1456182/roles/559942c4de1744169f8b5e0404ecde6b => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:45:31.405001 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ea8ebe-2ed8-4be6-baaa-d55e1ce710ba None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:31.405480 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ea8ebe-2ed8-4be6-baaa-d55e1ce710ba None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:31.405909 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ea8ebe-2ed8-4be6-baaa-d55e1ce710ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:31.428476 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 839/1681] 10.222.0.81 () {60 vars in 1143 bytes} [Wed Jul 20 22:45:31 2022] POST /identity/v3/users => generated 294 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:45:31.432368 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7a920a-55b8-49a3-9361-4a55761022eb admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:31.441687 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-e1ea8ebe-2ed8-4be6-baaa-d55e1ce710ba None None] MFA Rules not processed for user `d43fe9125f6243a796d0d98ef1456182`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:45:31.442927 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-e1ea8ebe-2ed8-4be6-baaa-d55e1ce710ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:31.443287 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7a920a-55b8-49a3-9361-4a55761022eb admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:31.506761 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c825c8d5-4e26-475d-8824-9a0a7cf3a015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:31.507656 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c825c8d5-4e26-475d-8824-9a0a7cf3a015 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:31.507949 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c825c8d5-4e26-475d-8824-9a0a7cf3a015 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:31.508232 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c825c8d5-4e26-475d-8824-9a0a7cf3a015 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:31.509085 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c825c8d5-4e26-475d-8824-9a0a7cf3a015 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:31.511038 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c825c8d5-4e26-475d-8824-9a0a7cf3a015 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:31.511140 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 843/1682] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:45:31 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:45:31.515227 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ea8ebe-2ed8-4be6-baaa-d55e1ce710ba None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:31.523246 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 840/1683] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:45:31 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 22:45:31.560843 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ea8ebe-2ed8-4be6-baaa-d55e1ce710ba None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:31.560843 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5cd182-de54-4ff0-b765-137e8bf8649c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:31.560843 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5cd182-de54-4ff0-b765-137e8bf8649c admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:31.560843 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5cd182-de54-4ff0-b765-137e8bf8649c admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:31.561104 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5cd182-de54-4ff0-b765-137e8bf8649c admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:31.561104 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5cd182-de54-4ff0-b765-137e8bf8649c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:31.561104 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5cd182-de54-4ff0-b765-137e8bf8649c admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:31.562832 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c825c8d5-4e26-475d-8824-9a0a7cf3a015 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:31.574447 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c825c8d5-4e26-475d-8824-9a0a7cf3a015 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:31.579741 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f5facd-3495-401d-b871-cdfbf8443b46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:31.580658 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f5facd-3495-401d-b871-cdfbf8443b46 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:31.580796 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f5facd-3495-401d-b871-cdfbf8443b46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:31.580979 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f5facd-3495-401d-b871-cdfbf8443b46 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:31.581708 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f5facd-3495-401d-b871-cdfbf8443b46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:31.583356 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f5facd-3495-401d-b871-cdfbf8443b46 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:31.594203 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 841/1684] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:45:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:45:31.599149 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f5facd-3495-401d-b871-cdfbf8443b46 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:31.609291 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f5facd-3495-401d-b871-cdfbf8443b46 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:31.614790 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce2fceb-b105-4456-8986-f95f93ecb0bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:31.615829 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce2fceb-b105-4456-8986-f95f93ecb0bd admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:31.616149 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce2fceb-b105-4456-8986-f95f93ecb0bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:31.616492 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce2fceb-b105-4456-8986-f95f93ecb0bd admin admin] PATH_INFO: `/v3/projects/9ab718882fdc4ec7af24826c3f1a8611/users/8b98b2ed85394635a3bed5541c5975d7/roles/559942c4de1744169f8b5e0404ecde6b` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:31.632994 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce2fceb-b105-4456-8986-f95f93ecb0bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ab718882fdc4ec7af24826c3f1a8611, user_id=8b98b2ed85394635a3bed5541c5975d7, role_id=559942c4de1744169f8b5e0404ecde6b)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:31.635404 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 844/1685] 10.222.0.81 () {60 vars in 1152 bytes} [Wed Jul 20 22:45:31 2022] POST /identity/v3/projects => generated 373 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 22:45:31.635970 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce2fceb-b105-4456-8986-f95f93ecb0bd admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:31.639943 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5cd182-de54-4ff0-b765-137e8bf8649c admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:31.656985 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:45:31.713236 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5cd182-de54-4ff0-b765-137e8bf8649c admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:31.718957 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be44d196-c5e6-46a6-8c6b-b1945bb81954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:31.720218 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be44d196-c5e6-46a6-8c6b-b1945bb81954 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:31.720511 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be44d196-c5e6-46a6-8c6b-b1945bb81954 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:31.720794 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be44d196-c5e6-46a6-8c6b-b1945bb81954 admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:31.721490 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be44d196-c5e6-46a6-8c6b-b1945bb81954 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:31.724586 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be44d196-c5e6-46a6-8c6b-b1945bb81954 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:31.738700 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 842/1686] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:45:31 2022] PUT /identity/v3/projects/9ab718882fdc4ec7af24826c3f1a8611/users/8b98b2ed85394635a3bed5541c5975d7/roles/559942c4de1744169f8b5e0404ecde6b => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 22:45:31.750075 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147d70ee-c013-4248-9d84-aae88931208f None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:31.750363 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147d70ee-c013-4248-9d84-aae88931208f None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:31.750668 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147d70ee-c013-4248-9d84-aae88931208f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:31.778543 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-147d70ee-c013-4248-9d84-aae88931208f None None] MFA Rules not processed for user `8b98b2ed85394635a3bed5541c5975d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:45:31.779927 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-147d70ee-c013-4248-9d84-aae88931208f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:31.841942 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 845/1687] 10.222.0.81 () {60 vars in 1143 bytes} [Wed Jul 20 22:45:31 2022] POST /identity/v3/users => generated 293 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:45:31.843708 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e3a30e-f0e7-4ca1-b3a3-6b47ab7ae4f1 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:31.844194 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e3a30e-f0e7-4ca1-b3a3-6b47ab7ae4f1 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:31.844557 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e3a30e-f0e7-4ca1-b3a3-6b47ab7ae4f1 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:31.846669 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 846/1688] 10.222.0.81 () {54 vars in 921 bytes} [Wed Jul 20 22:45:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:45:31.847801 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e3a30e-f0e7-4ca1-b3a3-6b47ab7ae4f1 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:31.851040 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 843/1689] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:45:31 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 22:45:31.855299 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147d70ee-c013-4248-9d84-aae88931208f None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:31.939508 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e3a30e-f0e7-4ca1-b3a3-6b47ab7ae4f1 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:31.944831 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147d70ee-c013-4248-9d84-aae88931208f None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:31.945236 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fbecf7-070d-4c98-9026-328b2aad5f9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:31.946870 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fbecf7-070d-4c98-9026-328b2aad5f9c admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:31.947857 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fbecf7-070d-4c98-9026-328b2aad5f9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:31.948032 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fbecf7-070d-4c98-9026-328b2aad5f9c admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:31.950482 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:45:31.951112 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fbecf7-070d-4c98-9026-328b2aad5f9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:31.953966 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fbecf7-070d-4c98-9026-328b2aad5f9c admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:31.955423 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc71454b-6145-422b-b020-33569926361f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:31.956220 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc71454b-6145-422b-b020-33569926361f admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:31.956605 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc71454b-6145-422b-b020-33569926361f admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:31.957004 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc71454b-6145-422b-b020-33569926361f admin admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:31.957766 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc71454b-6145-422b-b020-33569926361f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:31.960040 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc71454b-6145-422b-b020-33569926361f admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:31.967918 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 847/1690] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:45:31 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:45:31.969551 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd1801b-146c-45f9-b8dc-baf6f0c0552c None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:31.970550 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd1801b-146c-45f9-b8dc-baf6f0c0552c None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:31.970837 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd1801b-146c-45f9-b8dc-baf6f0c0552c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:31.996230 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-4cd1801b-146c-45f9-b8dc-baf6f0c0552c None None] MFA Rules not processed for user `d5500cebdd9d4f8e9aecb45d6c955878`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:45:31.997872 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-4cd1801b-146c-45f9-b8dc-baf6f0c0552c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:32.092571 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 844/1691] 10.222.0.81 () {60 vars in 1152 bytes} [Wed Jul 20 22:45:31 2022] POST /identity/v3/projects => generated 385 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 22:45:32.095755 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc71454b-6145-422b-b020-33569926361f admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:32.109437 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 848/1692] 10.222.0.81 () {58 vars in 1018 bytes} [Wed Jul 20 22:45:31 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:45:32.111384 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd1801b-146c-45f9-b8dc-baf6f0c0552c None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:32.128907 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd1801b-146c-45f9-b8dc-baf6f0c0552c None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:32.196087 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc71454b-6145-422b-b020-33569926361f admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:32.203981 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7097c6c5-277b-4458-92d3-0d26bd41aef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:32.205166 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7097c6c5-277b-4458-92d3-0d26bd41aef1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:32.205560 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7097c6c5-277b-4458-92d3-0d26bd41aef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:32.205945 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7097c6c5-277b-4458-92d3-0d26bd41aef1 admin admin] PATH_INFO: `/v3/projects/b6f7e409b62244ee97e6374c5ba54f6b/users/d1254f4f386543f8b91f8025ba1c6709/roles/d89b6c92d35a401a8517f1d47f6be843` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:32.229285 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7097c6c5-277b-4458-92d3-0d26bd41aef1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6f7e409b62244ee97e6374c5ba54f6b, user_id=d1254f4f386543f8b91f8025ba1c6709, role_id=d89b6c92d35a401a8517f1d47f6be843)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:32.232947 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf2a720-af78-47b3-899f-f95f917aaa27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:32.233701 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf2a720-af78-47b3-899f-f95f917aaa27 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:32.233811 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7097c6c5-277b-4458-92d3-0d26bd41aef1 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:32.233987 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf2a720-af78-47b3-899f-f95f917aaa27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:32.234272 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf2a720-af78-47b3-899f-f95f917aaa27 admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:32.235050 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf2a720-af78-47b3-899f-f95f917aaa27 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:32.237721 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf2a720-af78-47b3-899f-f95f917aaa27 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:32.411733 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 845/1693] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:45:32 2022] PUT /identity/v3/projects/b6f7e409b62244ee97e6374c5ba54f6b/users/d1254f4f386543f8b91f8025ba1c6709/roles/d89b6c92d35a401a8517f1d47f6be843 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:45:32.412507 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 849/1694] 10.222.0.81 () {60 vars in 1143 bytes} [Wed Jul 20 22:45:32 2022] POST /identity/v3/users => generated 293 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:45:32.412873 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7097c6c5-277b-4458-92d3-0d26bd41aef1 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:32.416046 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-7097c6c5-277b-4458-92d3-0d26bd41aef1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:32.426134 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf2a720-af78-47b3-899f-f95f917aaa27 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:32.430594 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-cbf2a720-af78-47b3-899f-f95f917aaa27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:32.566392 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7097c6c5-277b-4458-92d3-0d26bd41aef1 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:32.576255 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c834eae3-149e-421b-9ac3-078ad5484d18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:32.578044 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c834eae3-149e-421b-9ac3-078ad5484d18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:32.578223 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf2a720-af78-47b3-899f-f95f917aaa27 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:32.578728 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c834eae3-149e-421b-9ac3-078ad5484d18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:32.580587 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c834eae3-149e-421b-9ac3-078ad5484d18 admin admin] PATH_INFO: `/v3/users/3cd5c46e50714155aed3e9f439c2bf6d` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:32.582322 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c834eae3-149e-421b-9ac3-078ad5484d18 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3cd5c46e50714155aed3e9f439c2bf6d)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:32.585610 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c834eae3-149e-421b-9ac3-078ad5484d18 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:32.715180 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b9bad0-e685-4ed7-9bcb-d380bdd1fd63 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5500cebdd9d4f8e9aecb45d6c955878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:32.715985 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b9bad0-e685-4ed7-9bcb-d380bdd1fd63 service placement] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:32.716413 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b9bad0-e685-4ed7-9bcb-d380bdd1fd63 service placement] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:32.716820 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b9bad0-e685-4ed7-9bcb-d380bdd1fd63 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:32.731287 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b9bad0-e685-4ed7-9bcb-d380bdd1fd63 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:32.733247 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b9bad0-e685-4ed7-9bcb-d380bdd1fd63 service placement] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:32.748201 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 850/1695] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:45:32 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:45:32.763743 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b9bad0-e685-4ed7-9bcb-d380bdd1fd63 service placement] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:32.789657 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b9bad0-e685-4ed7-9bcb-d380bdd1fd63 service placement] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:32.801148 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f161f463-0e86-4b5d-a670-3a2408bd309d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:32.801958 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f161f463-0e86-4b5d-a670-3a2408bd309d admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:32.803338 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f161f463-0e86-4b5d-a670-3a2408bd309d admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:32.808203 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f161f463-0e86-4b5d-a670-3a2408bd309d admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:32.808203 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f161f463-0e86-4b5d-a670-3a2408bd309d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:32.808203 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f161f463-0e86-4b5d-a670-3a2408bd309d admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:32.835966 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 851/1696] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:45:32 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:45:32.839267 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f161f463-0e86-4b5d-a670-3a2408bd309d admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:32.849360 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f161f463-0e86-4b5d-a670-3a2408bd309d admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:32.856129 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f95b8c2-52fb-4c64-a652-9e975c85d8e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:32.858080 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f95b8c2-52fb-4c64-a652-9e975c85d8e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:32.858080 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f95b8c2-52fb-4c64-a652-9e975c85d8e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:32.858080 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f95b8c2-52fb-4c64-a652-9e975c85d8e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:32.858529 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f95b8c2-52fb-4c64-a652-9e975c85d8e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:32.860833 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f95b8c2-52fb-4c64-a652-9e975c85d8e6 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:32.864177 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-c834eae3-149e-421b-9ac3-078ad5484d18 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3cd5c46e50714155aed3e9f439c2bf6d'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:32.868069 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-c834eae3-149e-421b-9ac3-078ad5484d18 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3cd5c46e50714155aed3e9f439c2bf6d'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:32.869098 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-c834eae3-149e-421b-9ac3-078ad5484d18 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3cd5c46e50714155aed3e9f439c2bf6d'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:32.879198 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-c834eae3-149e-421b-9ac3-078ad5484d18 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3cd5c46e50714155aed3e9f439c2bf6d'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:32.880234 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 852/1697] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:45:32 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:45:32.885017 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f95b8c2-52fb-4c64-a652-9e975c85d8e6 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:32.890756 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-9f95b8c2-52fb-4c64-a652-9e975c85d8e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:32.913869 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f95b8c2-52fb-4c64-a652-9e975c85d8e6 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:32.919709 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67e7b55-cf47-4464-8a38-1ab2dd3e4b41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:32.920858 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67e7b55-cf47-4464-8a38-1ab2dd3e4b41 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:32.921291 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67e7b55-cf47-4464-8a38-1ab2dd3e4b41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:32.921676 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67e7b55-cf47-4464-8a38-1ab2dd3e4b41 admin admin] PATH_INFO: `/v3/projects/b6f7e409b62244ee97e6374c5ba54f6b/users/d1254f4f386543f8b91f8025ba1c6709/roles/d89b6c92d35a401a8517f1d47f6be843` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:32.924076 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67e7b55-cf47-4464-8a38-1ab2dd3e4b41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6f7e409b62244ee97e6374c5ba54f6b, user_id=d1254f4f386543f8b91f8025ba1c6709, role_id=d89b6c92d35a401a8517f1d47f6be843)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:32.926965 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67e7b55-cf47-4464-8a38-1ab2dd3e4b41 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:32.940280 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 853/1698] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:45:32 2022] PUT /identity/v3/projects/b6f7e409b62244ee97e6374c5ba54f6b/users/d1254f4f386543f8b91f8025ba1c6709/roles/d89b6c92d35a401a8517f1d47f6be843 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:45:32.942792 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67e7b55-cf47-4464-8a38-1ab2dd3e4b41 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:32.953099 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-d67e7b55-cf47-4464-8a38-1ab2dd3e4b41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:33.006774 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-c834eae3-149e-421b-9ac3-078ad5484d18 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3cd5c46e50714155aed3e9f439c2bf6d'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:33.046149 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 846/1699] 10.222.0.81 () {58 vars in 1223 bytes} [Wed Jul 20 22:45:32 2022] DELETE /identity/v3/users/3cd5c46e50714155aed3e9f439c2bf6d => generated 0 bytes in 630 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:45:33.048951 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc77ba74-70b0-4341-b2d3-307626a4e3bc None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:33.049249 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc77ba74-70b0-4341-b2d3-307626a4e3bc None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:33.053904 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc77ba74-70b0-4341-b2d3-307626a4e3bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:33.061315 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67e7b55-cf47-4464-8a38-1ab2dd3e4b41 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:33.108323 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-fc77ba74-70b0-4341-b2d3-307626a4e3bc None None] MFA Rules not processed for user `d1254f4f386543f8b91f8025ba1c6709`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:45:33.109749 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-fc77ba74-70b0-4341-b2d3-307626a4e3bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:33.154227 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698a62f2-e172-4cc8-a5c0-05862535ab9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:33.155224 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698a62f2-e172-4cc8-a5c0-05862535ab9e admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:33.158368 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698a62f2-e172-4cc8-a5c0-05862535ab9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:33.158776 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698a62f2-e172-4cc8-a5c0-05862535ab9e admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:33.159946 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698a62f2-e172-4cc8-a5c0-05862535ab9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:33.166413 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698a62f2-e172-4cc8-a5c0-05862535ab9e admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:33.194617 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:45:33.200885 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 854/1700] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:45:32 2022] GET /identity/v3/roles => generated 1420 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:45:33.206143 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698a62f2-e172-4cc8-a5c0-05862535ab9e admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:33.230073 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698a62f2-e172-4cc8-a5c0-05862535ab9e admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:33.247935 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97d2f30-6f33-4ee6-9aa5-16407746face admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:33.249674 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97d2f30-6f33-4ee6-9aa5-16407746face admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:33.250172 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97d2f30-6f33-4ee6-9aa5-16407746face admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:33.250557 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97d2f30-6f33-4ee6-9aa5-16407746face admin admin] PATH_INFO: `/v3/projects/9bab059a851f4964b3eb7c40e6e9c84a/users/ccc26e776c5c46de9dba4dd80b183080/roles/559942c4de1744169f8b5e0404ecde6b` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:33.261614 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:45:33.267985 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 847/1701] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:45:33 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:45:33.277495 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc77ba74-70b0-4341-b2d3-307626a4e3bc None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:33.279469 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-fc77ba74-70b0-4341-b2d3-307626a4e3bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:33.292496 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97d2f30-6f33-4ee6-9aa5-16407746face admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bab059a851f4964b3eb7c40e6e9c84a, user_id=ccc26e776c5c46de9dba4dd80b183080, role_id=559942c4de1744169f8b5e0404ecde6b)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:33.296002 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97d2f30-6f33-4ee6-9aa5-16407746face admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:33.354342 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc77ba74-70b0-4341-b2d3-307626a4e3bc None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:33.358310 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7178644-00c5-48be-a425-d2173769d3a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecf4db289fba4e4a806198c62d31f7c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:33.359264 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7178644-00c5-48be-a425-d2173769d3a9 service cinder] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:33.359623 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7178644-00c5-48be-a425-d2173769d3a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:33.359982 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7178644-00c5-48be-a425-d2173769d3a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:33.374750 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7178644-00c5-48be-a425-d2173769d3a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:33.376563 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7178644-00c5-48be-a425-d2173769d3a9 service cinder] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:33.390150 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 848/1702] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:45:33 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:45:33.411763 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 855/1703] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:45:33 2022] PUT /identity/v3/projects/9bab059a851f4964b3eb7c40e6e9c84a/users/ccc26e776c5c46de9dba4dd80b183080/roles/559942c4de1744169f8b5e0404ecde6b => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:45:33.417475 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c00378-c487-47d4-947f-d6e251b99745 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:33.417475 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c00378-c487-47d4-947f-d6e251b99745 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:33.417475 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c00378-c487-47d4-947f-d6e251b99745 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:33.464453 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-17c00378-c487-47d4-947f-d6e251b99745 None None] MFA Rules not processed for user `ccc26e776c5c46de9dba4dd80b183080`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:45:33.466180 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-17c00378-c487-47d4-947f-d6e251b99745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:33.604401 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 849/1704] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:45:33 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:45:33.609352 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97d2f30-6f33-4ee6-9aa5-16407746face admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:33.705628 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf851b8-093e-4594-9ea8-65eebf0478a4 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:33.709131 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf851b8-093e-4594-9ea8-65eebf0478a4 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:33.709131 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf851b8-093e-4594-9ea8-65eebf0478a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:33.801768 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-daf851b8-093e-4594-9ea8-65eebf0478a4 None None] MFA Rules not processed for user `0cf649be0c48492f8926f4436cbb3d6d`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:45:33.804906 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-daf851b8-093e-4594-9ea8-65eebf0478a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:33.833802 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97d2f30-6f33-4ee6-9aa5-16407746face admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:33.839627 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c78cce-fbfc-4c0f-889e-cf61d9f158ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:33.841174 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c78cce-fbfc-4c0f-889e-cf61d9f158ba admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:33.841464 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c78cce-fbfc-4c0f-889e-cf61d9f158ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:33.842882 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c78cce-fbfc-4c0f-889e-cf61d9f158ba admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:33.844103 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c78cce-fbfc-4c0f-889e-cf61d9f158ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:33.849138 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c78cce-fbfc-4c0f-889e-cf61d9f158ba admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:33.967052 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 850/1705] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:45:33 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:45:33.983709 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf851b8-093e-4594-9ea8-65eebf0478a4 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:33.991651 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-daf851b8-093e-4594-9ea8-65eebf0478a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:34.052609 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 856/1706] 10.222.0.81 () {60 vars in 1152 bytes} [Wed Jul 20 22:45:33 2022] POST /identity/v3/projects => generated 387 bytes in 441 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:45:34.053757 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c78cce-fbfc-4c0f-889e-cf61d9f158ba admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:34.241990 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf851b8-093e-4594-9ea8-65eebf0478a4 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:34.270759 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c78cce-fbfc-4c0f-889e-cf61d9f158ba admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:34.288957 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0499610-a550-41c3-9c1a-5395cd3957b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:34.289907 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0499610-a550-41c3-9c1a-5395cd3957b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:34.290346 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0499610-a550-41c3-9c1a-5395cd3957b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:34.290727 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0499610-a550-41c3-9c1a-5395cd3957b7 admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:34.299476 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0499610-a550-41c3-9c1a-5395cd3957b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:34.302449 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0499610-a550-41c3-9c1a-5395cd3957b7 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:34.394345 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50953be6-e731-4642-91e2-642b319a06a9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7829324db2574d38b18c749d0559177f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:34.395386 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50953be6-e731-4642-91e2-642b319a06a9 service nova] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:34.395678 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50953be6-e731-4642-91e2-642b319a06a9 service nova] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:34.395963 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50953be6-e731-4642-91e2-642b319a06a9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:34.443272 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 857/1707] 10.222.0.81 () {60 vars in 1143 bytes} [Wed Jul 20 22:45:34 2022] POST /identity/v3/users => generated 300 bytes in 390 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:45:34.444184 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0499610-a550-41c3-9c1a-5395cd3957b7 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:34.449836 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-a0499610-a550-41c3-9c1a-5395cd3957b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:34.483367 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0499610-a550-41c3-9c1a-5395cd3957b7 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:34.494215 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50953be6-e731-4642-91e2-642b319a06a9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:34.495881 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50953be6-e731-4642-91e2-642b319a06a9 service nova] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:34.498749 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35113391-e5de-4204-959a-e9d6d617067f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:34.499692 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35113391-e5de-4204-959a-e9d6d617067f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:34.499986 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35113391-e5de-4204-959a-e9d6d617067f admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:34.500267 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35113391-e5de-4204-959a-e9d6d617067f admin admin] PATH_INFO: `/v3/projects/8467d6d569584e26b7c4e1ceb3126598` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:34.505363 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35113391-e5de-4204-959a-e9d6d617067f admin admin] RBAC: Authorizing `identity:delete_project(project_id=8467d6d569584e26b7c4e1ceb3126598)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:34.506434 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 851/1708] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:45:33 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 523 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:45:34.507402 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50953be6-e731-4642-91e2-642b319a06a9 service nova] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:34.507591 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35113391-e5de-4204-959a-e9d6d617067f admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:34.518782 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50953be6-e731-4642-91e2-642b319a06a9 service nova] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:34.524599 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd05062-9f6a-4127-85dd-3339d6e66af4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:34.525298 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd05062-9f6a-4127-85dd-3339d6e66af4 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:34.529547 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd05062-9f6a-4127-85dd-3339d6e66af4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:34.529547 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd05062-9f6a-4127-85dd-3339d6e66af4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:34.529547 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd05062-9f6a-4127-85dd-3339d6e66af4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:34.529547 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd05062-9f6a-4127-85dd-3339d6e66af4 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:34.542093 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 852/1709] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:45:34 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:45:34.546675 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd05062-9f6a-4127-85dd-3339d6e66af4 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:34.560231 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd05062-9f6a-4127-85dd-3339d6e66af4 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:34.567173 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bc3287-d02e-4976-b5b5-02b134434f48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:34.569056 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bc3287-d02e-4976-b5b5-02b134434f48 admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:34.569538 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bc3287-d02e-4976-b5b5-02b134434f48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:34.569900 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bc3287-d02e-4976-b5b5-02b134434f48 admin admin] PATH_INFO: `/v3/projects/8b5b30e1cf174e0790e40c303ff7ba46/users/a6586620f2924b2c93d723af74cb3fac/roles/d89b6c92d35a401a8517f1d47f6be843` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:34.601095 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bc3287-d02e-4976-b5b5-02b134434f48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b5b30e1cf174e0790e40c303ff7ba46, user_id=a6586620f2924b2c93d723af74cb3fac, role_id=d89b6c92d35a401a8517f1d47f6be843)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:34.604531 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bc3287-d02e-4976-b5b5-02b134434f48 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:34.696216 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:45:34.706396 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 853/1710] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:45:34 2022] PUT /identity/v3/projects/8b5b30e1cf174e0790e40c303ff7ba46/users/a6586620f2924b2c93d723af74cb3fac/roles/d89b6c92d35a401a8517f1d47f6be843 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 22:45:34.711060 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bc3287-d02e-4976-b5b5-02b134434f48 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:34.729825 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-35113391-e5de-4204-959a-e9d6d617067f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8467d6d569584e26b7c4e1ceb3126598', 'request_id': 'req-35113391-e5de-4204-959a-e9d6d617067f'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:34.751067 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-35113391-e5de-4204-959a-e9d6d617067f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8467d6d569584e26b7c4e1ceb3126598', 'request_id': 'req-35113391-e5de-4204-959a-e9d6d617067f'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:34.858672 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bc3287-d02e-4976-b5b5-02b134434f48 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:34.860839 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-35113391-e5de-4204-959a-e9d6d617067f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8467d6d569584e26b7c4e1ceb3126598', 'request_id': 'req-35113391-e5de-4204-959a-e9d6d617067f'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:34.876570 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-35113391-e5de-4204-959a-e9d6d617067f admin admin] The token cache is being invalidate because project 8467d6d569584e26b7c4e1ceb3126598 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:45:34.876868 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-35113391-e5de-4204-959a-e9d6d617067f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:34.880152 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-35113391-e5de-4204-959a-e9d6d617067f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:34.889430 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 858/1711] 10.222.0.81 () {58 vars in 1232 bytes} [Wed Jul 20 22:45:34 2022] DELETE /identity/v3/projects/8467d6d569584e26b7c4e1ceb3126598 => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:45:34.895396 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35113391-e5de-4204-959a-e9d6d617067f admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:34.898915 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-35113391-e5de-4204-959a-e9d6d617067f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:35.067037 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318b59b6-3ecb-4bca-bee0-cbf0228d5da1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:35.067739 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318b59b6-3ecb-4bca-bee0-cbf0228d5da1 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:35.069394 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318b59b6-3ecb-4bca-bee0-cbf0228d5da1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:35.069679 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318b59b6-3ecb-4bca-bee0-cbf0228d5da1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:35.071292 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318b59b6-3ecb-4bca-bee0-cbf0228d5da1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:35.075635 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318b59b6-3ecb-4bca-bee0-cbf0228d5da1 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:35.109380 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 854/1712] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:45:34 2022] GET /identity/v3/roles => generated 1420 bytes in 400 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:45:35.111779 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318b59b6-3ecb-4bca-bee0-cbf0228d5da1 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:35.116725 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-318b59b6-3ecb-4bca-bee0-cbf0228d5da1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:35.121980 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35113391-e5de-4204-959a-e9d6d617067f admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:35.138871 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572f3121-dc31-4100-889f-2439b2f78b08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:35.144240 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572f3121-dc31-4100-889f-2439b2f78b08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:35.144240 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572f3121-dc31-4100-889f-2439b2f78b08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:35.144240 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572f3121-dc31-4100-889f-2439b2f78b08 admin admin] PATH_INFO: `/v3/users/fa9199f34c1048ed8e756b4fa324e606` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:35.165552 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572f3121-dc31-4100-889f-2439b2f78b08 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa9199f34c1048ed8e756b4fa324e606)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:35.174103 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572f3121-dc31-4100-889f-2439b2f78b08 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:35.200149 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318b59b6-3ecb-4bca-bee0-cbf0228d5da1 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:35.214009 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c10c62-6175-4323-b4e7-9f607e981563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:35.215657 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c10c62-6175-4323-b4e7-9f607e981563 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:35.215957 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c10c62-6175-4323-b4e7-9f607e981563 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:35.216240 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c10c62-6175-4323-b4e7-9f607e981563 admin admin] PATH_INFO: `/v3/users/d43fe9125f6243a796d0d98ef1456182` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:35.220735 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c10c62-6175-4323-b4e7-9f607e981563 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d43fe9125f6243a796d0d98ef1456182)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:35.226682 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c10c62-6175-4323-b4e7-9f607e981563 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:35.655990 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-572f3121-dc31-4100-889f-2439b2f78b08 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa9199f34c1048ed8e756b4fa324e606'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:35.657583 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-572f3121-dc31-4100-889f-2439b2f78b08 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa9199f34c1048ed8e756b4fa324e606'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:35.658405 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-572f3121-dc31-4100-889f-2439b2f78b08 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa9199f34c1048ed8e756b4fa324e606'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:35.670109 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-572f3121-dc31-4100-889f-2439b2f78b08 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa9199f34c1048ed8e756b4fa324e606'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:35.670295 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-c3c10c62-6175-4323-b4e7-9f607e981563 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd43fe9125f6243a796d0d98ef1456182'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:35.671166 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-c3c10c62-6175-4323-b4e7-9f607e981563 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd43fe9125f6243a796d0d98ef1456182'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:35.673700 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-c3c10c62-6175-4323-b4e7-9f607e981563 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd43fe9125f6243a796d0d98ef1456182'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:35.689704 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-c3c10c62-6175-4323-b4e7-9f607e981563 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd43fe9125f6243a796d0d98ef1456182'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:35.790033 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-572f3121-dc31-4100-889f-2439b2f78b08 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa9199f34c1048ed8e756b4fa324e606'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:35.841822 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 859/1713] 10.222.0.81 () {58 vars in 1223 bytes} [Wed Jul 20 22:45:34 2022] DELETE /identity/v3/users/fa9199f34c1048ed8e756b4fa324e606 => generated 0 bytes in 950 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:45:35.843381 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572f3121-dc31-4100-889f-2439b2f78b08 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:35.845980 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-572f3121-dc31-4100-889f-2439b2f78b08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:35.901732 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:DELETE=5,keystone:SELECT=93,keystone:INSERT=2 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:45:35.961967 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-c3c10c62-6175-4323-b4e7-9f607e981563 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd43fe9125f6243a796d0d98ef1456182'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:36.031871 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 855/1714] 10.222.0.81 () {58 vars in 1223 bytes} [Wed Jul 20 22:45:35 2022] DELETE /identity/v3/users/d43fe9125f6243a796d0d98ef1456182 => generated 0 bytes in 922 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:45:36.054908 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc09e3ce-2fce-4111-8561-fc36542f7a9f None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:36.055237 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc09e3ce-2fce-4111-8561-fc36542f7a9f None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:36.055526 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc09e3ce-2fce-4111-8561-fc36542f7a9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:36.108453 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-dc09e3ce-2fce-4111-8561-fc36542f7a9f None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:45:36.110382 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572f3121-dc31-4100-889f-2439b2f78b08 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:36.112870 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-dc09e3ce-2fce-4111-8561-fc36542f7a9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:36.269000 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:45:36.318982 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee16af59-13a6-490b-bcac-05813deb0c42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:36.322674 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee16af59-13a6-490b-bcac-05813deb0c42 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:36.323042 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee16af59-13a6-490b-bcac-05813deb0c42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:36.323337 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee16af59-13a6-490b-bcac-05813deb0c42 admin admin] PATH_INFO: `/v3/projects/8b5b30e1cf174e0790e40c303ff7ba46/users/a6586620f2924b2c93d723af74cb3fac/roles/d89b6c92d35a401a8517f1d47f6be843` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:36.342289 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee16af59-13a6-490b-bcac-05813deb0c42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b5b30e1cf174e0790e40c303ff7ba46, user_id=a6586620f2924b2c93d723af74cb3fac, role_id=d89b6c92d35a401a8517f1d47f6be843)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:36.348897 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee16af59-13a6-490b-bcac-05813deb0c42 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:36.390944 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 860/1715] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:45:35 2022] PUT /identity/v3/projects/8b5b30e1cf174e0790e40c303ff7ba46/users/a6586620f2924b2c93d723af74cb3fac/roles/d89b6c92d35a401a8517f1d47f6be843 => generated 0 bytes in 548 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:45:36.405993 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 856/1716] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:45:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 369 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:45:36.406719 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde03856-809c-4208-b257-b9eb869eecd7 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:36.409120 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c2547c-2a51-487c-be2b-fd4b2d383d1e None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:36.409418 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c2547c-2a51-487c-be2b-fd4b2d383d1e None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:36.410985 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde03856-809c-4208-b257-b9eb869eecd7 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:36.411400 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde03856-809c-4208-b257-b9eb869eecd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:36.412969 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c2547c-2a51-487c-be2b-fd4b2d383d1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:36.473778 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-fde03856-809c-4208-b257-b9eb869eecd7 None None] MFA Rules not processed for user `a6586620f2924b2c93d723af74cb3fac`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:45:36.475316 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-fde03856-809c-4208-b257-b9eb869eecd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:36.512164 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-03c2547c-2a51-487c-be2b-fd4b2d383d1e None None] MFA Rules not processed for user `1ed9249b2a9b42d9972ed71c04f2c888`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:45:36.513685 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-03c2547c-2a51-487c-be2b-fd4b2d383d1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:36.673226 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 857/1717] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:45:36 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:45:36.684580 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde03856-809c-4208-b257-b9eb869eecd7 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:36.692480 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-fde03856-809c-4208-b257-b9eb869eecd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:36.707187 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 861/1718] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:45:36 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:45:36.708325 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c2547c-2a51-487c-be2b-fd4b2d383d1e None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:36.709794 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-03c2547c-2a51-487c-be2b-fd4b2d383d1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:36.875035 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde03856-809c-4208-b257-b9eb869eecd7 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:36.900460 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4911dd89-3981-49f7-94c5-9e2136191939 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:36.900851 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c2547c-2a51-487c-be2b-fd4b2d383d1e None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:36.901679 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4911dd89-3981-49f7-94c5-9e2136191939 service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:36.902560 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4911dd89-3981-49f7-94c5-9e2136191939 service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:36.903376 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4911dd89-3981-49f7-94c5-9e2136191939 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:36.908513 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab66978a-c860-40d7-98c0-19be52a521ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecf4db289fba4e4a806198c62d31f7c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:36.909313 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab66978a-c860-40d7-98c0-19be52a521ff service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:36.910392 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab66978a-c860-40d7-98c0-19be52a521ff service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:36.910392 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab66978a-c860-40d7-98c0-19be52a521ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:36.945489 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab66978a-c860-40d7-98c0-19be52a521ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:36.947769 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab66978a-c860-40d7-98c0-19be52a521ff service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:36.987040 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 862/1719] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:45:36 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:45:37.014722 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4437b6c5-1989-474b-8542-33fe83851e0e None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:37.017700 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4437b6c5-1989-474b-8542-33fe83851e0e None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:37.020751 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4437b6c5-1989-474b-8542-33fe83851e0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:37.100260 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-4437b6c5-1989-474b-8542-33fe83851e0e None None] MFA Rules not processed for user `1ed9249b2a9b42d9972ed71c04f2c888`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:45:37.118803 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4911dd89-3981-49f7-94c5-9e2136191939 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:37.121327 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4911dd89-3981-49f7-94c5-9e2136191939 service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:37.130795 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-4437b6c5-1989-474b-8542-33fe83851e0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:37.160346 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 863/1720] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:45:36 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:45:37.168665 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4437b6c5-1989-474b-8542-33fe83851e0e None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:37.170492 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-4437b6c5-1989-474b-8542-33fe83851e0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:37.172467 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 858/1721] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:45:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 496 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:45:37.210390 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4911dd89-3981-49f7-94c5-9e2136191939 service neutron] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:37.210390 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-4911dd89-3981-49f7-94c5-9e2136191939 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:37.234674 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4911dd89-3981-49f7-94c5-9e2136191939 service neutron] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:37.259205 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601c0e80-0b62-4071-9c91-90ce5b14ad0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:37.260308 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601c0e80-0b62-4071-9c91-90ce5b14ad0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:37.261042 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601c0e80-0b62-4071-9c91-90ce5b14ad0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:37.261042 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601c0e80-0b62-4071-9c91-90ce5b14ad0b admin admin] PATH_INFO: `/v3/projects/39aa4cce7199466fbd878465214e4137` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:37.263125 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601c0e80-0b62-4071-9c91-90ce5b14ad0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=39aa4cce7199466fbd878465214e4137)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:37.265120 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601c0e80-0b62-4071-9c91-90ce5b14ad0b admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:37.454482 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4437b6c5-1989-474b-8542-33fe83851e0e None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:37.468802 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32f3829-e938-43c5-95d9-1c6952a84147 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7829324db2574d38b18c749d0559177f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:37.469774 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32f3829-e938-43c5-95d9-1c6952a84147 service nova] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:37.471778 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32f3829-e938-43c5-95d9-1c6952a84147 service nova] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:37.472264 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32f3829-e938-43c5-95d9-1c6952a84147 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:37.494168 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32f3829-e938-43c5-95d9-1c6952a84147 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:37.503374 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32f3829-e938-43c5-95d9-1c6952a84147 service nova] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:37.517989 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 864/1722] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:45:37 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 22:45:37.544905 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-601c0e80-0b62-4071-9c91-90ce5b14ad0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39aa4cce7199466fbd878465214e4137', 'request_id': 'req-601c0e80-0b62-4071-9c91-90ce5b14ad0b'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:37.718594 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-601c0e80-0b62-4071-9c91-90ce5b14ad0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39aa4cce7199466fbd878465214e4137', 'request_id': 'req-601c0e80-0b62-4071-9c91-90ce5b14ad0b'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:37.725843 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-601c0e80-0b62-4071-9c91-90ce5b14ad0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39aa4cce7199466fbd878465214e4137', 'request_id': 'req-601c0e80-0b62-4071-9c91-90ce5b14ad0b'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:37.769694 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-601c0e80-0b62-4071-9c91-90ce5b14ad0b admin admin] The token cache is being invalidate because project 39aa4cce7199466fbd878465214e4137 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:45:37.770601 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-601c0e80-0b62-4071-9c91-90ce5b14ad0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:37.771712 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-601c0e80-0b62-4071-9c91-90ce5b14ad0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:37.786605 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 859/1723] 10.222.0.81 () {58 vars in 1232 bytes} [Wed Jul 20 22:45:37 2022] DELETE /identity/v3/projects/39aa4cce7199466fbd878465214e4137 => generated 0 bytes in 577 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:45:37.789458 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f87935b-b07c-4e12-9dec-e32fffd95349 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:37.789458 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f87935b-b07c-4e12-9dec-e32fffd95349 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:37.789952 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f87935b-b07c-4e12-9dec-e32fffd95349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:37.821325 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-4f87935b-b07c-4e12-9dec-e32fffd95349 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:45:37.829301 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-4f87935b-b07c-4e12-9dec-e32fffd95349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:37.927290 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:45:38.046681 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72c198a-f9d4-48f9-bb33-a54b06fada3b None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:38.046972 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72c198a-f9d4-48f9-bb33-a54b06fada3b None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:38.047358 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72c198a-f9d4-48f9-bb33-a54b06fada3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:38.051236 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 865/1724] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:45:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:45:38.087411 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-e72c198a-f9d4-48f9-bb33-a54b06fada3b None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:45:38.088698 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-e72c198a-f9d4-48f9-bb33-a54b06fada3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:38.139521 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 860/1725] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:45:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:45:38.149988 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f87935b-b07c-4e12-9dec-e32fffd95349 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:38.171806 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f87935b-b07c-4e12-9dec-e32fffd95349 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:38.200323 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aca2905-b8e6-4fc2-a023-edbc280814ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:38.201287 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aca2905-b8e6-4fc2-a023-edbc280814ba admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:38.201803 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aca2905-b8e6-4fc2-a023-edbc280814ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:38.202308 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aca2905-b8e6-4fc2-a023-edbc280814ba admin admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:38.203367 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aca2905-b8e6-4fc2-a023-edbc280814ba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:45:38.203918 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aca2905-b8e6-4fc2-a023-edbc280814ba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:38.205780 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aca2905-b8e6-4fc2-a023-edbc280814ba admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:38.244332 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 866/1726] 10.222.0.81 () {58 vars in 1152 bytes} [Wed Jul 20 22:45:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:45:38.249082 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72c198a-f9d4-48f9-bb33-a54b06fada3b None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:38.259346 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72c198a-f9d4-48f9-bb33-a54b06fada3b None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:38.277332 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fffa82-6e84-4ba0-9be2-88a59344c562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:38.278211 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fffa82-6e84-4ba0-9be2-88a59344c562 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:38.278953 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fffa82-6e84-4ba0-9be2-88a59344c562 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:38.281085 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fffa82-6e84-4ba0-9be2-88a59344c562 admin admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:38.282531 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fffa82-6e84-4ba0-9be2-88a59344c562 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:38.293619 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fffa82-6e84-4ba0-9be2-88a59344c562 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:38.528140 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aca2905-b8e6-4fc2-a023-edbc280814ba admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:38.528358 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 861/1727] 10.222.0.81 () {60 vars in 1152 bytes} [Wed Jul 20 22:45:38 2022] POST /identity/v3/projects => generated 371 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:45:38.560433 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aca2905-b8e6-4fc2-a023-edbc280814ba admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:38.563268 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fffa82-6e84-4ba0-9be2-88a59344c562 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:38.573304 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-74fffa82-6e84-4ba0-9be2-88a59344c562 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:38.653583 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149717c2-8b1f-460d-b9d7-790726917b27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:38.654631 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149717c2-8b1f-460d-b9d7-790726917b27 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:38.655173 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149717c2-8b1f-460d-b9d7-790726917b27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:38.655854 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149717c2-8b1f-460d-b9d7-790726917b27 admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:38.657496 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149717c2-8b1f-460d-b9d7-790726917b27 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:38.660174 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149717c2-8b1f-460d-b9d7-790726917b27 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:38.701634 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:45:38.791207 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fffa82-6e84-4ba0-9be2-88a59344c562 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:38.801746 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9334c6-7ac1-41e8-9b15-f6c813736a82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:38.802932 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9334c6-7ac1-41e8-9b15-f6c813736a82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:38.803449 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9334c6-7ac1-41e8-9b15-f6c813736a82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:38.803910 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9334c6-7ac1-41e8-9b15-f6c813736a82 admin admin] PATH_INFO: `/v3/users/906771f309044b178b9218d79390e35d` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:38.808287 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9334c6-7ac1-41e8-9b15-f6c813736a82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=906771f309044b178b9218d79390e35d)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:38.810529 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9334c6-7ac1-41e8-9b15-f6c813736a82 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:38.818201 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 867/1728] 10.222.0.81 () {60 vars in 1143 bytes} [Wed Jul 20 22:45:38 2022] POST /identity/v3/users => generated 286 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:45:38.830936 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149717c2-8b1f-460d-b9d7-790726917b27 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:38.862011 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149717c2-8b1f-460d-b9d7-790726917b27 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:38.872472 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40af2c47-bf77-47e3-932a-1395bc80f6e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:38.873437 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40af2c47-bf77-47e3-932a-1395bc80f6e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:38.873875 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40af2c47-bf77-47e3-932a-1395bc80f6e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:38.875711 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40af2c47-bf77-47e3-932a-1395bc80f6e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:38.877017 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40af2c47-bf77-47e3-932a-1395bc80f6e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:38.888895 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40af2c47-bf77-47e3-932a-1395bc80f6e1 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:38.905324 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 868/1729] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:45:38 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:45:38.919324 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40af2c47-bf77-47e3-932a-1395bc80f6e1 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:38.934722 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40af2c47-bf77-47e3-932a-1395bc80f6e1 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:38.944455 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8494638e-aa71-4887-b127-4b182873b74f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:38.946376 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8494638e-aa71-4887-b127-4b182873b74f admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:38.946376 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8494638e-aa71-4887-b127-4b182873b74f admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:38.946376 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8494638e-aa71-4887-b127-4b182873b74f admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:38.947769 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8494638e-aa71-4887-b127-4b182873b74f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:38.950869 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8494638e-aa71-4887-b127-4b182873b74f admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:39.021265 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 869/1730] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:45:38 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:45:39.022248 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8494638e-aa71-4887-b127-4b182873b74f admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:39.060354 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8494638e-aa71-4887-b127-4b182873b74f admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:39.074824 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f9a39d-a7e8-4a58-a75b-2c715e783543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:39.075812 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f9a39d-a7e8-4a58-a75b-2c715e783543 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:39.076104 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f9a39d-a7e8-4a58-a75b-2c715e783543 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:39.076387 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f9a39d-a7e8-4a58-a75b-2c715e783543 admin admin] PATH_INFO: `/v3/projects/cf14585020c44f3593c109e0164639c4/users/3f77f7db11ad43c1ae237c06dcc8baa0/roles/559942c4de1744169f8b5e0404ecde6b` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:39.102185 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f9a39d-a7e8-4a58-a75b-2c715e783543 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf14585020c44f3593c109e0164639c4, user_id=3f77f7db11ad43c1ae237c06dcc8baa0, role_id=559942c4de1744169f8b5e0404ecde6b)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:39.105004 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f9a39d-a7e8-4a58-a75b-2c715e783543 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:39.183207 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-ba9334c6-7ac1-41e8-9b15-f6c813736a82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '906771f309044b178b9218d79390e35d'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:39.188442 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-ba9334c6-7ac1-41e8-9b15-f6c813736a82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '906771f309044b178b9218d79390e35d'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:39.191056 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-ba9334c6-7ac1-41e8-9b15-f6c813736a82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '906771f309044b178b9218d79390e35d'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:39.215543 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-ba9334c6-7ac1-41e8-9b15-f6c813736a82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '906771f309044b178b9218d79390e35d'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:39.224633 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 870/1731] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:45:39 2022] PUT /identity/v3/projects/cf14585020c44f3593c109e0164639c4/users/3f77f7db11ad43c1ae237c06dcc8baa0/roles/559942c4de1744169f8b5e0404ecde6b => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:45:39.248916 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14afebe5-2a33-463b-8bf5-a7283da03928 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:39.255396 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14afebe5-2a33-463b-8bf5-a7283da03928 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:39.255396 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14afebe5-2a33-463b-8bf5-a7283da03928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:39.344034 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-14afebe5-2a33-463b-8bf5-a7283da03928 None None] MFA Rules not processed for user `3f77f7db11ad43c1ae237c06dcc8baa0`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:45:39.347967 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-ba9334c6-7ac1-41e8-9b15-f6c813736a82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '906771f309044b178b9218d79390e35d'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:39.366021 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-14afebe5-2a33-463b-8bf5-a7283da03928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:39.400296 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 862/1732] 10.222.0.81 () {58 vars in 1223 bytes} [Wed Jul 20 22:45:38 2022] DELETE /identity/v3/users/906771f309044b178b9218d79390e35d => generated 0 bytes in 838 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 22:45:39.407981 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b53a8b8-6a1c-4edd-bcb9-11d9c2fdf2cd None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:39.407981 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b53a8b8-6a1c-4edd-bcb9-11d9c2fdf2cd None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:39.412399 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b53a8b8-6a1c-4edd-bcb9-11d9c2fdf2cd None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:39.417638 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 863/1733] 10.222.0.81 () {54 vars in 912 bytes} [Wed Jul 20 22:45:39 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:45:39.422297 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bec9e3b-9ae9-47bf-841a-b5c22bda083d None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:39.422879 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bec9e3b-9ae9-47bf-841a-b5c22bda083d None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:39.423506 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bec9e3b-9ae9-47bf-841a-b5c22bda083d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:39.516152 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-6bec9e3b-9ae9-47bf-841a-b5c22bda083d None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:45:39.518367 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-6bec9e3b-9ae9-47bf-841a-b5c22bda083d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:39.601677 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 871/1734] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:45:39 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 358 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:45:39.607589 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f66923-db8f-41ad-b139-6df88b8dcf53 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:39.611644 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f66923-db8f-41ad-b139-6df88b8dcf53 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:39.612194 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f66923-db8f-41ad-b139-6df88b8dcf53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:39.691875 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 864/1735] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:45:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:45:39.693735 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bec9e3b-9ae9-47bf-841a-b5c22bda083d None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:39.696082 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-6bec9e3b-9ae9-47bf-841a-b5c22bda083d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:39.730072 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-56f66923-db8f-41ad-b139-6df88b8dcf53 None None] MFA Rules not processed for user `333c2b1b11cc4c478abfd965a72e9d66`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:45:39.733157 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-56f66923-db8f-41ad-b139-6df88b8dcf53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:39.805533 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bec9e3b-9ae9-47bf-841a-b5c22bda083d None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:39.815771 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbdfd5a-fd3f-4705-b322-a1fcf6e6469b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecf4db289fba4e4a806198c62d31f7c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:39.816697 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbdfd5a-fd3f-4705-b322-a1fcf6e6469b service cinder] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:39.822558 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbdfd5a-fd3f-4705-b322-a1fcf6e6469b service cinder] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:39.822558 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbdfd5a-fd3f-4705-b322-a1fcf6e6469b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:39.822558 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-abbdfd5a-fd3f-4705-b322-a1fcf6e6469b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:39.865686 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 872/1736] 10.222.0.81 () {58 vars in 1009 bytes} [Wed Jul 20 22:45:39 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:45:39.866361 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f66923-db8f-41ad-b139-6df88b8dcf53 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:39.874464 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-56f66923-db8f-41ad-b139-6df88b8dcf53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:39.914182 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f66923-db8f-41ad-b139-6df88b8dcf53 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:39.929486 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7725ae7b-691e-4a8c-bc25-27544f5360cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:39.930183 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7725ae7b-691e-4a8c-bc25-27544f5360cb admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:39.930470 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7725ae7b-691e-4a8c-bc25-27544f5360cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:39.930749 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7725ae7b-691e-4a8c-bc25-27544f5360cb admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:39.931477 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7725ae7b-691e-4a8c-bc25-27544f5360cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:39.933573 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7725ae7b-691e-4a8c-bc25-27544f5360cb admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:39.942358 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:45:39.978739 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbdfd5a-fd3f-4705-b322-a1fcf6e6469b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:39.980496 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbdfd5a-fd3f-4705-b322-a1fcf6e6469b service cinder] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:39.991702 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 865/1737] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:45:39 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:45:40.003625 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbdfd5a-fd3f-4705-b322-a1fcf6e6469b service cinder] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:40.008386 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-abbdfd5a-fd3f-4705-b322-a1fcf6e6469b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:40.037466 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbdfd5a-fd3f-4705-b322-a1fcf6e6469b service cinder] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:40.050620 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86be464b-2dcc-4da3-ae32-05945dc48a30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:40.051918 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86be464b-2dcc-4da3-ae32-05945dc48a30 service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:40.052214 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86be464b-2dcc-4da3-ae32-05945dc48a30 service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:40.052536 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86be464b-2dcc-4da3-ae32-05945dc48a30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:40.079508 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86be464b-2dcc-4da3-ae32-05945dc48a30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:40.080972 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86be464b-2dcc-4da3-ae32-05945dc48a30 service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:40.093059 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 873/1738] 10.222.0.81 () {60 vars in 1152 bytes} [Wed Jul 20 22:45:39 2022] POST /identity/v3/projects => generated 373 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:45:40.096366 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7725ae7b-691e-4a8c-bc25-27544f5360cb admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:40.103984 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 866/1739] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:45:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:45:40.104813 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86be464b-2dcc-4da3-ae32-05945dc48a30 service neutron] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:40.130956 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86be464b-2dcc-4da3-ae32-05945dc48a30 service neutron] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:40.225922 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7725ae7b-691e-4a8c-bc25-27544f5360cb admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:40.236959 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec6b53e-de04-4e32-86cb-0e7d96cbd28a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:40.237646 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec6b53e-de04-4e32-86cb-0e7d96cbd28a service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:40.237932 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec6b53e-de04-4e32-86cb-0e7d96cbd28a service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:40.238261 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec6b53e-de04-4e32-86cb-0e7d96cbd28a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:40.256642 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec6b53e-de04-4e32-86cb-0e7d96cbd28a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:40.263181 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec6b53e-de04-4e32-86cb-0e7d96cbd28a service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:40.281508 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 874/1740] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:45:40 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:45:40.310375 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a248642-4d30-46f4-b63f-9ccf5c21c491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:40.311410 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a248642-4d30-46f4-b63f-9ccf5c21c491 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:40.311705 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a248642-4d30-46f4-b63f-9ccf5c21c491 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:40.311985 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a248642-4d30-46f4-b63f-9ccf5c21c491 admin admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:40.312697 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a248642-4d30-46f4-b63f-9ccf5c21c491 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:40.315211 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a248642-4d30-46f4-b63f-9ccf5c21c491 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:40.417373 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 867/1741] 10.222.0.81 () {60 vars in 1143 bytes} [Wed Jul 20 22:45:40 2022] POST /identity/v3/users => generated 293 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:45:40.422802 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec6b53e-de04-4e32-86cb-0e7d96cbd28a service neutron] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:40.443494 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a248642-4d30-46f4-b63f-9ccf5c21c491 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:40.443663 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec6b53e-de04-4e32-86cb-0e7d96cbd28a service neutron] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:40.445639 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-6a248642-4d30-46f4-b63f-9ccf5c21c491 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:40.455933 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd49be3-ec3e-4e5a-a9ee-9b23812beb23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:40.456706 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd49be3-ec3e-4e5a-a9ee-9b23812beb23 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:40.456920 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd49be3-ec3e-4e5a-a9ee-9b23812beb23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:40.457198 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd49be3-ec3e-4e5a-a9ee-9b23812beb23 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:40.460006 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd49be3-ec3e-4e5a-a9ee-9b23812beb23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:40.463201 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd49be3-ec3e-4e5a-a9ee-9b23812beb23 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:40.468005 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a248642-4d30-46f4-b63f-9ccf5c21c491 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:40.485262 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b354a0-6c88-40cd-b5fb-b3b3e0d3c81d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:40.486230 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b354a0-6c88-40cd-b5fb-b3b3e0d3c81d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:40.486522 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b354a0-6c88-40cd-b5fb-b3b3e0d3c81d admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:40.486965 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b354a0-6c88-40cd-b5fb-b3b3e0d3c81d admin admin] PATH_INFO: `/v3/projects/c66dad1ba27644a5ae96c9c6080f46d6` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:40.491308 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b354a0-6c88-40cd-b5fb-b3b3e0d3c81d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c66dad1ba27644a5ae96c9c6080f46d6)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:40.492592 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 875/1742] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:45:40 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:45:40.493235 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b354a0-6c88-40cd-b5fb-b3b3e0d3c81d admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:40.498902 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd49be3-ec3e-4e5a-a9ee-9b23812beb23 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:40.511223 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:45:40.519257 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd49be3-ec3e-4e5a-a9ee-9b23812beb23 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:40.547590 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa363833-ba0b-410b-84b2-bfcdd83aee4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:40.548548 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa363833-ba0b-410b-84b2-bfcdd83aee4d admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:40.548836 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa363833-ba0b-410b-84b2-bfcdd83aee4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:40.549142 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa363833-ba0b-410b-84b2-bfcdd83aee4d admin admin] PATH_INFO: `/v3/projects/36f0533b022d49479b649b0f832eacab/users/98febf1fdc0a40f28111b5034e83ca97/roles/d89b6c92d35a401a8517f1d47f6be843` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:40.566156 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa363833-ba0b-410b-84b2-bfcdd83aee4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=36f0533b022d49479b649b0f832eacab, user_id=98febf1fdc0a40f28111b5034e83ca97, role_id=d89b6c92d35a401a8517f1d47f6be843)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:40.569203 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa363833-ba0b-410b-84b2-bfcdd83aee4d admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:40.663306 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 876/1743] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:45:40 2022] PUT /identity/v3/projects/36f0533b022d49479b649b0f832eacab/users/98febf1fdc0a40f28111b5034e83ca97/roles/d89b6c92d35a401a8517f1d47f6be843 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:45:40.664856 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e68f1ed-25f7-4ed4-84f9-ea20d8de46ab None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:40.666884 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e68f1ed-25f7-4ed4-84f9-ea20d8de46ab None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:40.667740 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e68f1ed-25f7-4ed4-84f9-ea20d8de46ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:40.678055 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-59b354a0-6c88-40cd-b5fb-b3b3e0d3c81d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c66dad1ba27644a5ae96c9c6080f46d6', 'request_id': 'req-59b354a0-6c88-40cd-b5fb-b3b3e0d3c81d'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:40.732066 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-4e68f1ed-25f7-4ed4-84f9-ea20d8de46ab None None] MFA Rules not processed for user `3e072e59bf3e46aaa062ec85b0e2c91a`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:45:40.734479 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-4e68f1ed-25f7-4ed4-84f9-ea20d8de46ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:40.763705 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-59b354a0-6c88-40cd-b5fb-b3b3e0d3c81d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c66dad1ba27644a5ae96c9c6080f46d6', 'request_id': 'req-59b354a0-6c88-40cd-b5fb-b3b3e0d3c81d'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:40.778749 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-59b354a0-6c88-40cd-b5fb-b3b3e0d3c81d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c66dad1ba27644a5ae96c9c6080f46d6', 'request_id': 'req-59b354a0-6c88-40cd-b5fb-b3b3e0d3c81d'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:40.821769 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-59b354a0-6c88-40cd-b5fb-b3b3e0d3c81d admin admin] The token cache is being invalidate because project c66dad1ba27644a5ae96c9c6080f46d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:45:40.824084 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-59b354a0-6c88-40cd-b5fb-b3b3e0d3c81d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:40.825992 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-59b354a0-6c88-40cd-b5fb-b3b3e0d3c81d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:40.831410 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 868/1744] 10.222.0.81 () {58 vars in 1232 bytes} [Wed Jul 20 22:45:40 2022] DELETE /identity/v3/projects/c66dad1ba27644a5ae96c9c6080f46d6 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:45:40.832507 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b354a0-6c88-40cd-b5fb-b3b3e0d3c81d admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:40.840226 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-59b354a0-6c88-40cd-b5fb-b3b3e0d3c81d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:40.864854 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 877/1745] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:45:40 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:45:40.868280 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23e9f06-a007-4218-964e-4631de2c28c5 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:40.868415 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23e9f06-a007-4218-964e-4631de2c28c5 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:40.868925 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23e9f06-a007-4218-964e-4631de2c28c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:40.918239 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-e23e9f06-a007-4218-964e-4631de2c28c5 None None] MFA Rules not processed for user `1b458e3837be425bba1f880895918287`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:45:40.921179 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-e23e9f06-a007-4218-964e-4631de2c28c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:40.998845 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 878/1746] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:45:40 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:45:41.000542 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23e9f06-a007-4218-964e-4631de2c28c5 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:41.004635 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b354a0-6c88-40cd-b5fb-b3b3e0d3c81d admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:41.009957 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-e23e9f06-a007-4218-964e-4631de2c28c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:41.018735 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62eb8bca-e78b-4311-80f5-051bf7914c3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:41.019791 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eb8bca-e78b-4311-80f5-051bf7914c3b admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:41.020253 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eb8bca-e78b-4311-80f5-051bf7914c3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:41.020715 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eb8bca-e78b-4311-80f5-051bf7914c3b admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:41.022900 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62eb8bca-e78b-4311-80f5-051bf7914c3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:41.025990 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62eb8bca-e78b-4311-80f5-051bf7914c3b admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:41.039987 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23e9f06-a007-4218-964e-4631de2c28c5 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:41.040856 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 869/1747] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:45:40 2022] GET /identity/v3/roles => generated 1420 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:45:41.041917 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62eb8bca-e78b-4311-80f5-051bf7914c3b admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:41.044199 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-62eb8bca-e78b-4311-80f5-051bf7914c3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:41.053297 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fcb180-3ff9-4dab-be9e-e213b134edd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:41.055070 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fcb180-3ff9-4dab-be9e-e213b134edd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:41.055479 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fcb180-3ff9-4dab-be9e-e213b134edd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:41.055649 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fcb180-3ff9-4dab-be9e-e213b134edd8 admin admin] PATH_INFO: `/v3/users/d1254f4f386543f8b91f8025ba1c6709` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:41.057302 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fcb180-3ff9-4dab-be9e-e213b134edd8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1254f4f386543f8b91f8025ba1c6709)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:41.063756 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fcb180-3ff9-4dab-be9e-e213b134edd8 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:41.184533 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62eb8bca-e78b-4311-80f5-051bf7914c3b admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:41.191208 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a6ca45-9ef4-400c-9c52-8b46e9bdf4ea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7829324db2574d38b18c749d0559177f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:41.191885 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a6ca45-9ef4-400c-9c52-8b46e9bdf4ea service nova] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:41.192224 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a6ca45-9ef4-400c-9c52-8b46e9bdf4ea service nova] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:41.192514 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a6ca45-9ef4-400c-9c52-8b46e9bdf4ea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:41.196409 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-a5a6ca45-9ef4-400c-9c52-8b46e9bdf4ea service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:41.211671 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a6ca45-9ef4-400c-9c52-8b46e9bdf4ea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:41.211671 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a6ca45-9ef4-400c-9c52-8b46e9bdf4ea service nova] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:41.226163 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 870/1748] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:45:41 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:45:41.241198 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a32aac-809c-41d5-b3e4-86301133c27a None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:41.242077 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a32aac-809c-41d5-b3e4-86301133c27a None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:41.242467 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a32aac-809c-41d5-b3e4-86301133c27a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:41.288720 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-d2a32aac-809c-41d5-b3e4-86301133c27a None None] MFA Rules not processed for user `1b458e3837be425bba1f880895918287`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:45:41.292688 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-d2a32aac-809c-41d5-b3e4-86301133c27a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:41.313136 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 871/1749] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:45:41 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:45:41.313136 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a32aac-809c-41d5-b3e4-86301133c27a None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:41.369096 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a32aac-809c-41d5-b3e4-86301133c27a None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:41.387719 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e3622b-8972-4049-917b-b51ef540a0d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:41.387719 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e3622b-8972-4049-917b-b51ef540a0d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:41.387719 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e3622b-8972-4049-917b-b51ef540a0d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:41.387719 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e3622b-8972-4049-917b-b51ef540a0d2 admin admin] PATH_INFO: `/v3/projects/36f0533b022d49479b649b0f832eacab/users/98febf1fdc0a40f28111b5034e83ca97/roles/d89b6c92d35a401a8517f1d47f6be843` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:41.393569 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-b2fcb180-3ff9-4dab-be9e-e213b134edd8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1254f4f386543f8b91f8025ba1c6709'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:41.393905 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e3622b-8972-4049-917b-b51ef540a0d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=36f0533b022d49479b649b0f832eacab, user_id=98febf1fdc0a40f28111b5034e83ca97, role_id=d89b6c92d35a401a8517f1d47f6be843)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:41.393992 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-b2fcb180-3ff9-4dab-be9e-e213b134edd8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1254f4f386543f8b91f8025ba1c6709'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:41.393992 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-b2fcb180-3ff9-4dab-be9e-e213b134edd8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1254f4f386543f8b91f8025ba1c6709'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:41.394113 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e3622b-8972-4049-917b-b51ef540a0d2 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:41.400126 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-b2fcb180-3ff9-4dab-be9e-e213b134edd8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1254f4f386543f8b91f8025ba1c6709'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:41.411115 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 872/1750] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:45:41 2022] PUT /identity/v3/projects/36f0533b022d49479b649b0f832eacab/users/98febf1fdc0a40f28111b5034e83ca97/roles/d89b6c92d35a401a8517f1d47f6be843 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:45:41.412260 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e3622b-8972-4049-917b-b51ef540a0d2 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:41.414094 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-b0e3622b-8972-4049-917b-b51ef540a0d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:41.494086 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-b2fcb180-3ff9-4dab-be9e-e213b134edd8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1254f4f386543f8b91f8025ba1c6709'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:41.542317 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 879/1751] 10.222.0.81 () {58 vars in 1223 bytes} [Wed Jul 20 22:45:40 2022] DELETE /identity/v3/users/d1254f4f386543f8b91f8025ba1c6709 => generated 0 bytes in 540 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:45:41.545955 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fcb180-3ff9-4dab-be9e-e213b134edd8 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:41.551801 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-b2fcb180-3ff9-4dab-be9e-e213b134edd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:41.590728 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e3622b-8972-4049-917b-b51ef540a0d2 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:41.745898 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fcb180-3ff9-4dab-be9e-e213b134edd8 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:41.759389 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8bd638-6aa3-4116-9df6-2e2e555fba3a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5500cebdd9d4f8e9aecb45d6c955878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:41.759876 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8bd638-6aa3-4116-9df6-2e2e555fba3a service placement] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:41.760907 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca51f8a-654a-4273-9d08-51d2228bae9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:41.761146 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8bd638-6aa3-4116-9df6-2e2e555fba3a service placement] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:41.762264 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8bd638-6aa3-4116-9df6-2e2e555fba3a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:41.763412 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca51f8a-654a-4273-9d08-51d2228bae9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:41.763702 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca51f8a-654a-4273-9d08-51d2228bae9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:41.763987 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca51f8a-654a-4273-9d08-51d2228bae9d admin admin] PATH_INFO: `/v3/projects/10dcd3a5465d434cbb63e588e4d4df8c` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:41.765832 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca51f8a-654a-4273-9d08-51d2228bae9d admin admin] RBAC: Authorizing `identity:delete_project(project_id=10dcd3a5465d434cbb63e588e4d4df8c)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:41.769461 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-cc8bd638-6aa3-4116-9df6-2e2e555fba3a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:41.772388 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca51f8a-654a-4273-9d08-51d2228bae9d admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:41.797217 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8bd638-6aa3-4116-9df6-2e2e555fba3a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:41.798888 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8bd638-6aa3-4116-9df6-2e2e555fba3a service placement] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:41.826520 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 880/1752] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:45:41 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:45:41.827939 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f535181-c61d-4780-98ec-2a398cb1441f None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:41.828237 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f535181-c61d-4780-98ec-2a398cb1441f None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:41.828521 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f535181-c61d-4780-98ec-2a398cb1441f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:41.914557 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-bca51f8a-654a-4273-9d08-51d2228bae9d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10dcd3a5465d434cbb63e588e4d4df8c', 'request_id': 'req-bca51f8a-654a-4273-9d08-51d2228bae9d'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:41.921825 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:45:41.923761 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-7f535181-c61d-4780-98ec-2a398cb1441f None None] MFA Rules not processed for user `98febf1fdc0a40f28111b5034e83ca97`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:45:41.925133 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-7f535181-c61d-4780-98ec-2a398cb1441f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:42.004505 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-bca51f8a-654a-4273-9d08-51d2228bae9d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10dcd3a5465d434cbb63e588e4d4df8c', 'request_id': 'req-bca51f8a-654a-4273-9d08-51d2228bae9d'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:42.015602 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-bca51f8a-654a-4273-9d08-51d2228bae9d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10dcd3a5465d434cbb63e588e4d4df8c', 'request_id': 'req-bca51f8a-654a-4273-9d08-51d2228bae9d'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:42.049633 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-bca51f8a-654a-4273-9d08-51d2228bae9d admin admin] The token cache is being invalidate because project 10dcd3a5465d434cbb63e588e4d4df8c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:45:42.050015 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-bca51f8a-654a-4273-9d08-51d2228bae9d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:42.051666 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-bca51f8a-654a-4273-9d08-51d2228bae9d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:42.057177 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 873/1753] 10.222.0.81 () {58 vars in 1232 bytes} [Wed Jul 20 22:45:41 2022] DELETE /identity/v3/projects/10dcd3a5465d434cbb63e588e4d4df8c => generated 0 bytes in 645 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:45:42.058173 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca51f8a-654a-4273-9d08-51d2228bae9d admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:42.063149 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-bca51f8a-654a-4273-9d08-51d2228bae9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:42.095964 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 881/1754] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:45:41 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:45:42.098127 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21be2d81-3b92-44ad-97f6-004206510c1a None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:42.107185 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21be2d81-3b92-44ad-97f6-004206510c1a None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:42.107185 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21be2d81-3b92-44ad-97f6-004206510c1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:42.177098 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-21be2d81-3b92-44ad-97f6-004206510c1a None None] MFA Rules not processed for user `1ed9249b2a9b42d9972ed71c04f2c888`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:45:42.177098 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-21be2d81-3b92-44ad-97f6-004206510c1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:42.242712 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca51f8a-654a-4273-9d08-51d2228bae9d admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:42.257603 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e822766-57d3-4290-976e-404761d3c867 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7829324db2574d38b18c749d0559177f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:42.258290 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e822766-57d3-4290-976e-404761d3c867 service nova] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:42.258583 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e822766-57d3-4290-976e-404761d3c867 service nova] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:42.258866 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e822766-57d3-4290-976e-404761d3c867 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:42.263364 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-5e822766-57d3-4290-976e-404761d3c867 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:42.323339 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 882/1755] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:45:42 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:45:42.325133 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00c9969-3f59-4233-b95c-f912afef83c6 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:42.326257 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00c9969-3f59-4233-b95c-f912afef83c6 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:42.326553 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00c9969-3f59-4233-b95c-f912afef83c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:42.368601 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-a00c9969-3f59-4233-b95c-f912afef83c6 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:45:42.370088 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-a00c9969-3f59-4233-b95c-f912afef83c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:42.415819 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e822766-57d3-4290-976e-404761d3c867 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:42.418027 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e822766-57d3-4290-976e-404761d3c867 service nova] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:42.430075 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 874/1756] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:45:42 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:45:42.431001 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e822766-57d3-4290-976e-404761d3c867 service nova] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:42.432516 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-5e822766-57d3-4290-976e-404761d3c867 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:42.590027 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 883/1757] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:45:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:45:42.590027 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2f6b7d-0f18-49a3-8cf9-d403fa87037d None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:42.590535 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2f6b7d-0f18-49a3-8cf9-d403fa87037d None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:42.590771 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2f6b7d-0f18-49a3-8cf9-d403fa87037d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:42.636368 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e822766-57d3-4290-976e-404761d3c867 service nova] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:42.647451 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d3d6d3-200e-49df-970a-05a870d85780 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecf4db289fba4e4a806198c62d31f7c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:42.650051 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d3d6d3-200e-49df-970a-05a870d85780 service cinder] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:42.651141 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d3d6d3-200e-49df-970a-05a870d85780 service cinder] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:42.651612 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d3d6d3-200e-49df-970a-05a870d85780 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:42.654251 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-0d2f6b7d-0f18-49a3-8cf9-d403fa87037d None None] MFA Rules not processed for user `0cf649be0c48492f8926f4436cbb3d6d`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:45:42.660434 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-0d2f6b7d-0f18-49a3-8cf9-d403fa87037d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:42.678325 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d3d6d3-200e-49df-970a-05a870d85780 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:42.683603 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d3d6d3-200e-49df-970a-05a870d85780 service cinder] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:42.713271 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:45:42.713384 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 875/1758] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:45:42 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:45:42.714140 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e61b69-590c-41b4-a674-4f66e0ae6bb8 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:42.714428 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e61b69-590c-41b4-a674-4f66e0ae6bb8 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:42.722187 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e61b69-590c-41b4-a674-4f66e0ae6bb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:42.808460 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-b2e61b69-590c-41b4-a674-4f66e0ae6bb8 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:45:42.809879 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-b2e61b69-590c-41b4-a674-4f66e0ae6bb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:42.859263 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 876/1759] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:45:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:45:42.862837 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e61b69-590c-41b4-a674-4f66e0ae6bb8 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:42.870408 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 884/1760] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:45:42 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:45:42.890847 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e61b69-590c-41b4-a674-4f66e0ae6bb8 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:42.919757 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2672343a-1c6a-480d-82ba-dc4691685a17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:42.921172 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2672343a-1c6a-480d-82ba-dc4691685a17 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:42.923039 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2672343a-1c6a-480d-82ba-dc4691685a17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:42.924830 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2672343a-1c6a-480d-82ba-dc4691685a17 admin admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:42.930997 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2672343a-1c6a-480d-82ba-dc4691685a17 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:45:42.932631 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2672343a-1c6a-480d-82ba-dc4691685a17 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:42.934456 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2672343a-1c6a-480d-82ba-dc4691685a17 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:42.978669 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 877/1761] 10.222.0.81 () {58 vars in 1152 bytes} [Wed Jul 20 22:45:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:45:42.984405 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2f6b7d-0f18-49a3-8cf9-d403fa87037d None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:43.001853 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2f6b7d-0f18-49a3-8cf9-d403fa87037d None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:43.016344 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd083a5a-c4b8-4106-9eae-5dd45bcf80e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:43.017294 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd083a5a-c4b8-4106-9eae-5dd45bcf80e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:43.017750 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd083a5a-c4b8-4106-9eae-5dd45bcf80e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:43.018179 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd083a5a-c4b8-4106-9eae-5dd45bcf80e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:43.019855 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd083a5a-c4b8-4106-9eae-5dd45bcf80e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:43.022239 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd083a5a-c4b8-4106-9eae-5dd45bcf80e3 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:43.102758 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2672343a-1c6a-480d-82ba-dc4691685a17 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:43.119046 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2672343a-1c6a-480d-82ba-dc4691685a17 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:43.136387 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e258ef1-3d09-47d4-8670-9083c30294c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecf4db289fba4e4a806198c62d31f7c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:43.137144 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e258ef1-3d09-47d4-8670-9083c30294c6 service cinder] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:43.142758 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e258ef1-3d09-47d4-8670-9083c30294c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:43.143075 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e258ef1-3d09-47d4-8670-9083c30294c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:43.146732 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-2e258ef1-3d09-47d4-8670-9083c30294c6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:43.147373 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 885/1762] 10.222.0.81 () {60 vars in 1152 bytes} [Wed Jul 20 22:45:42 2022] POST /identity/v3/projects => generated 371 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:45:43.148631 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd083a5a-c4b8-4106-9eae-5dd45bcf80e3 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:43.153584 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-fd083a5a-c4b8-4106-9eae-5dd45bcf80e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:43.339296 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e258ef1-3d09-47d4-8670-9083c30294c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:43.345998 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e258ef1-3d09-47d4-8670-9083c30294c6 service cinder] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:43.355838 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd083a5a-c4b8-4106-9eae-5dd45bcf80e3 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:43.373703 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 878/1763] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:45:43 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:45:43.377189 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e258ef1-3d09-47d4-8670-9083c30294c6 service cinder] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:43.378065 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b262e5-1518-4b03-997f-be659f4fcd48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:43.383127 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b262e5-1518-4b03-997f-be659f4fcd48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:43.383127 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b262e5-1518-4b03-997f-be659f4fcd48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:43.383127 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b262e5-1518-4b03-997f-be659f4fcd48 admin admin] PATH_INFO: `/v3/projects/b6f7e409b62244ee97e6374c5ba54f6b` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:43.383127 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b262e5-1518-4b03-997f-be659f4fcd48 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6f7e409b62244ee97e6374c5ba54f6b)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:43.386725 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b262e5-1518-4b03-997f-be659f4fcd48 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:43.401099 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e258ef1-3d09-47d4-8670-9083c30294c6 service cinder] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:43.416095 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed21ae6-29de-4083-a608-1dd7e5bb4c2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:43.418184 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed21ae6-29de-4083-a608-1dd7e5bb4c2f admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:43.418616 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed21ae6-29de-4083-a608-1dd7e5bb4c2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:43.419134 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed21ae6-29de-4083-a608-1dd7e5bb4c2f admin admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:43.422081 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed21ae6-29de-4083-a608-1dd7e5bb4c2f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:43.427430 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed21ae6-29de-4083-a608-1dd7e5bb4c2f admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:43.639264 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 879/1764] 10.222.0.81 () {60 vars in 1143 bytes} [Wed Jul 20 22:45:43 2022] POST /identity/v3/users => generated 286 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:45:43.641275 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac318655-d488-4996-a5bf-58aad966224f None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:43.641275 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac318655-d488-4996-a5bf-58aad966224f None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:43.641751 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac318655-d488-4996-a5bf-58aad966224f None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:43.647061 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 880/1765] 10.222.0.81 () {54 vars in 912 bytes} [Wed Jul 20 22:45:43 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:45:43.648004 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac318655-d488-4996-a5bf-58aad966224f None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:43.674025 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-a0b262e5-1518-4b03-997f-be659f4fcd48 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6f7e409b62244ee97e6374c5ba54f6b', 'request_id': 'req-a0b262e5-1518-4b03-997f-be659f4fcd48'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:43.705774 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-a0b262e5-1518-4b03-997f-be659f4fcd48 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6f7e409b62244ee97e6374c5ba54f6b', 'request_id': 'req-a0b262e5-1518-4b03-997f-be659f4fcd48'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:43.799057 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-a0b262e5-1518-4b03-997f-be659f4fcd48 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6f7e409b62244ee97e6374c5ba54f6b', 'request_id': 'req-a0b262e5-1518-4b03-997f-be659f4fcd48'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:43.812948 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac318655-d488-4996-a5bf-58aad966224f None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:43.820549 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8931312c-b9b2-46cb-b64f-62f73fa13b8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:43.821522 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8931312c-b9b2-46cb-b64f-62f73fa13b8f admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:43.821956 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8931312c-b9b2-46cb-b64f-62f73fa13b8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:43.822232 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-a0b262e5-1518-4b03-997f-be659f4fcd48 admin admin] The token cache is being invalidate because project b6f7e409b62244ee97e6374c5ba54f6b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:45:43.822539 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8931312c-b9b2-46cb-b64f-62f73fa13b8f admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:43.822705 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-a0b262e5-1518-4b03-997f-be659f4fcd48 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:43.823709 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8931312c-b9b2-46cb-b64f-62f73fa13b8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:43.825578 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-a0b262e5-1518-4b03-997f-be659f4fcd48 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:43.826863 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8931312c-b9b2-46cb-b64f-62f73fa13b8f admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:43.833418 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 886/1766] 10.222.0.81 () {58 vars in 1232 bytes} [Wed Jul 20 22:45:43 2022] DELETE /identity/v3/projects/b6f7e409b62244ee97e6374c5ba54f6b => generated 0 bytes in 685 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:45:43.835681 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fc10b6-b372-4bd1-8d03-f29e6f310f5b None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:43.837930 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fc10b6-b372-4bd1-8d03-f29e6f310f5b None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:43.839305 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fc10b6-b372-4bd1-8d03-f29e6f310f5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:43.847457 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 881/1767] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:45:43 2022] GET /identity/v3/roles => generated 1420 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:45:43.849848 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2931c746-ac9c-408d-8b96-5a234b573956 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:43.850293 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2931c746-ac9c-408d-8b96-5a234b573956 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:43.851395 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2931c746-ac9c-408d-8b96-5a234b573956 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:43.929952 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-2931c746-ac9c-408d-8b96-5a234b573956 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:45:43.936683 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-2931c746-ac9c-408d-8b96-5a234b573956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:44.007242 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 882/1768] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:45:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:45:44.010086 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2931c746-ac9c-408d-8b96-5a234b573956 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:44.011155 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-81fc10b6-b372-4bd1-8d03-f29e6f310f5b None None] MFA Rules not processed for user `333c2b1b11cc4c478abfd965a72e9d66`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:45:44.012634 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-81fc10b6-b372-4bd1-8d03-f29e6f310f5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:44.049802 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-2931c746-ac9c-408d-8b96-5a234b573956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:44.099614 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2931c746-ac9c-408d-8b96-5a234b573956 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:44.116032 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f912ba9-d8d5-4beb-8b22-7b7a3d8e8dc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:44.118128 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f912ba9-d8d5-4beb-8b22-7b7a3d8e8dc2 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:44.118425 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f912ba9-d8d5-4beb-8b22-7b7a3d8e8dc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:44.118689 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f912ba9-d8d5-4beb-8b22-7b7a3d8e8dc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:44.119476 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f912ba9-d8d5-4beb-8b22-7b7a3d8e8dc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:44.127353 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f912ba9-d8d5-4beb-8b22-7b7a3d8e8dc2 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:44.164911 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 883/1769] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:45:44 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:45:44.166631 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee2c7a1-30b6-4f0f-987d-db05a3831d20 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:44.167149 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee2c7a1-30b6-4f0f-987d-db05a3831d20 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:44.167579 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee2c7a1-30b6-4f0f-987d-db05a3831d20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:44.212334 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 887/1770] 10.222.0.81 () {58 vars in 1009 bytes} [Wed Jul 20 22:45:43 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 378 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:45:44.219404 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fc10b6-b372-4bd1-8d03-f29e6f310f5b None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:44.264976 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-7ee2c7a1-30b6-4f0f-987d-db05a3831d20 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:45:44.266970 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-7ee2c7a1-30b6-4f0f-987d-db05a3831d20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:44.297451 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 884/1771] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:45:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:45:44.300973 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee2c7a1-30b6-4f0f-987d-db05a3831d20 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:44.302528 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fc10b6-b372-4bd1-8d03-f29e6f310f5b None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:44.305505 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-7ee2c7a1-30b6-4f0f-987d-db05a3831d20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:44.348350 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee2c7a1-30b6-4f0f-987d-db05a3831d20 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:44.360107 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2da70fc-e1f5-4be5-8b65-6a7245e923c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:44.360395 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8e964e-4aa5-40df-b9ee-94124fae14cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:44.361453 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8e964e-4aa5-40df-b9ee-94124fae14cf admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:44.361745 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8e964e-4aa5-40df-b9ee-94124fae14cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:44.362133 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8e964e-4aa5-40df-b9ee-94124fae14cf admin admin] PATH_INFO: `/v3/projects/22824ce78e8a4fc79f8394e536768ab3/users/531e1e88ccf94f91a058f7b1334836a6/roles/559942c4de1744169f8b5e0404ecde6b` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:44.364939 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2da70fc-e1f5-4be5-8b65-6a7245e923c4 service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:44.365502 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2da70fc-e1f5-4be5-8b65-6a7245e923c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:44.365986 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2da70fc-e1f5-4be5-8b65-6a7245e923c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:44.387254 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2da70fc-e1f5-4be5-8b65-6a7245e923c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:44.397893 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2da70fc-e1f5-4be5-8b65-6a7245e923c4 service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:44.430557 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 885/1772] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:45:44 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:45:44.444081 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8e964e-4aa5-40df-b9ee-94124fae14cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=22824ce78e8a4fc79f8394e536768ab3, user_id=531e1e88ccf94f91a058f7b1334836a6, role_id=559942c4de1744169f8b5e0404ecde6b)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:44.444966 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2da70fc-e1f5-4be5-8b65-6a7245e923c4 service neutron] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:44.460689 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8e964e-4aa5-40df-b9ee-94124fae14cf admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:44.482836 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2da70fc-e1f5-4be5-8b65-6a7245e923c4 service neutron] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:44.518885 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04de0acf-7400-4247-8b0b-da66af8cc14f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:44.520343 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04de0acf-7400-4247-8b0b-da66af8cc14f admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:44.522682 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04de0acf-7400-4247-8b0b-da66af8cc14f admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:44.523229 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04de0acf-7400-4247-8b0b-da66af8cc14f admin admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:44.524146 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04de0acf-7400-4247-8b0b-da66af8cc14f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:45:44.526672 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04de0acf-7400-4247-8b0b-da66af8cc14f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:44.532372 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04de0acf-7400-4247-8b0b-da66af8cc14f admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:44.532637 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:45:44.538430 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 888/1773] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:45:44 2022] PUT /identity/v3/projects/22824ce78e8a4fc79f8394e536768ab3/users/531e1e88ccf94f91a058f7b1334836a6/roles/559942c4de1744169f8b5e0404ecde6b => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:45:44.549503 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6b980d-d1c3-4cde-8dfb-4acbee87e93b None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:44.549798 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6b980d-d1c3-4cde-8dfb-4acbee87e93b None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:44.550077 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6b980d-d1c3-4cde-8dfb-4acbee87e93b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:44.570655 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:45:44.573609 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 886/1774] 10.222.0.81 () {58 vars in 1152 bytes} [Wed Jul 20 22:45:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:45:44.575227 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04de0acf-7400-4247-8b0b-da66af8cc14f admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:44.600340 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04de0acf-7400-4247-8b0b-da66af8cc14f admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:44.665393 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-3c6b980d-d1c3-4cde-8dfb-4acbee87e93b None None] MFA Rules not processed for user `531e1e88ccf94f91a058f7b1334836a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:45:44.672589 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-3c6b980d-d1c3-4cde-8dfb-4acbee87e93b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:44.819790 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824e7490-9b05-478e-8c62-a0049baed949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:44.820488 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824e7490-9b05-478e-8c62-a0049baed949 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:44.820780 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824e7490-9b05-478e-8c62-a0049baed949 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:44.823179 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824e7490-9b05-478e-8c62-a0049baed949 admin admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:44.823179 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824e7490-9b05-478e-8c62-a0049baed949 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:44.823857 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824e7490-9b05-478e-8c62-a0049baed949 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:44.902074 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 889/1775] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:45:44 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 340 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:45:45.030197 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 887/1776] 10.222.0.81 () {60 vars in 1152 bytes} [Wed Jul 20 22:45:44 2022] POST /identity/v3/projects => generated 379 bytes in 456 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:45:45.034502 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6b980d-d1c3-4cde-8dfb-4acbee87e93b None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:45.051987 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6b980d-d1c3-4cde-8dfb-4acbee87e93b None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:45.243631 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b899a43-e2eb-4092-aa4a-cdd73ee7cbae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:45.247636 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b899a43-e2eb-4092-aa4a-cdd73ee7cbae admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:45.247984 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b899a43-e2eb-4092-aa4a-cdd73ee7cbae admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:45.248311 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b899a43-e2eb-4092-aa4a-cdd73ee7cbae admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:45.251791 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b899a43-e2eb-4092-aa4a-cdd73ee7cbae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:45.262200 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b899a43-e2eb-4092-aa4a-cdd73ee7cbae admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:45.473109 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:45:45.596820 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 890/1777] 10.222.0.81 () {60 vars in 1143 bytes} [Wed Jul 20 22:45:45 2022] POST /identity/v3/users => generated 290 bytes in 562 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 22:45:45.599842 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824e7490-9b05-478e-8c62-a0049baed949 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:45.617086 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824e7490-9b05-478e-8c62-a0049baed949 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:45.637110 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e654df1-9146-427f-a745-b9442891c149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:45.637110 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e654df1-9146-427f-a745-b9442891c149 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:45.637110 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e654df1-9146-427f-a745-b9442891c149 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:45.637110 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e654df1-9146-427f-a745-b9442891c149 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:45.637830 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e654df1-9146-427f-a745-b9442891c149 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:45.639523 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e654df1-9146-427f-a745-b9442891c149 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:45.677540 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b899a43-e2eb-4092-aa4a-cdd73ee7cbae admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:45.688798 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 888/1778] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:45:45 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:45:45.692216 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b899a43-e2eb-4092-aa4a-cdd73ee7cbae admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:45.704248 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2af63f-e3ab-4d94-a1f2-0ce513b8338d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:45.708330 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2af63f-e3ab-4d94-a1f2-0ce513b8338d admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:45.708844 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2af63f-e3ab-4d94-a1f2-0ce513b8338d admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:45.709333 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2af63f-e3ab-4d94-a1f2-0ce513b8338d admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:45.710232 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2af63f-e3ab-4d94-a1f2-0ce513b8338d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:45.714300 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2af63f-e3ab-4d94-a1f2-0ce513b8338d admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:45.753366 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e654df1-9146-427f-a745-b9442891c149 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:45.754038 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 891/1779] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:45:45 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 22:45:45.769291 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e654df1-9146-427f-a745-b9442891c149 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:45.804024 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f6e27a-486a-4bf5-9764-6b5ae5780cf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:45.804959 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f6e27a-486a-4bf5-9764-6b5ae5780cf6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:45.805253 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f6e27a-486a-4bf5-9764-6b5ae5780cf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:45.805541 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f6e27a-486a-4bf5-9764-6b5ae5780cf6 admin admin] PATH_INFO: `/v3/projects/44d624a90e2a4e4196a0ebdd778b566e/users/903fa5c85a8047c1ad504413b77e4f28/roles/559942c4de1744169f8b5e0404ecde6b` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:45.828366 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2af63f-e3ab-4d94-a1f2-0ce513b8338d admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:45.830333 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-cd2af63f-e3ab-4d94-a1f2-0ce513b8338d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:45.843176 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f6e27a-486a-4bf5-9764-6b5ae5780cf6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44d624a90e2a4e4196a0ebdd778b566e, user_id=903fa5c85a8047c1ad504413b77e4f28, role_id=559942c4de1744169f8b5e0404ecde6b)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:45.852008 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f6e27a-486a-4bf5-9764-6b5ae5780cf6 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:45.961393 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 889/1780] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:45:45 2022] PUT /identity/v3/projects/44d624a90e2a4e4196a0ebdd778b566e/users/903fa5c85a8047c1ad504413b77e4f28/roles/559942c4de1744169f8b5e0404ecde6b => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:45:45.973381 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c50d14-4120-462a-a84b-7eeeac5be115 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:45.973700 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c50d14-4120-462a-a84b-7eeeac5be115 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:45.973958 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c50d14-4120-462a-a84b-7eeeac5be115 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:45.992637 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2af63f-e3ab-4d94-a1f2-0ce513b8338d admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:46.061006 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-18c50d14-4120-462a-a84b-7eeeac5be115 None None] MFA Rules not processed for user `903fa5c85a8047c1ad504413b77e4f28`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:45:46.062397 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-18c50d14-4120-462a-a84b-7eeeac5be115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:46.085042 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0fb75a-e0cb-47c9-9a95-0a0d4dee734b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7829324db2574d38b18c749d0559177f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:46.085733 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0fb75a-e0cb-47c9-9a95-0a0d4dee734b service nova] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:46.086026 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0fb75a-e0cb-47c9-9a95-0a0d4dee734b service nova] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:46.086310 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0fb75a-e0cb-47c9-9a95-0a0d4dee734b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:46.161008 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0fb75a-e0cb-47c9-9a95-0a0d4dee734b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:46.162749 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0fb75a-e0cb-47c9-9a95-0a0d4dee734b service nova] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:46.173661 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 892/1781] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:45:45 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 22:45:46.236167 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 890/1782] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:45:45 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:45:46.237313 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0fb75a-e0cb-47c9-9a95-0a0d4dee734b service nova] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:46.399219 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0fb75a-e0cb-47c9-9a95-0a0d4dee734b service nova] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:46.415748 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2952fa-936e-4472-afec-197f29eb239d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:46.416898 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2952fa-936e-4472-afec-197f29eb239d admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:46.417194 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2952fa-936e-4472-afec-197f29eb239d admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:46.417476 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2952fa-936e-4472-afec-197f29eb239d admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:46.418193 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2952fa-936e-4472-afec-197f29eb239d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:46.420619 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2952fa-936e-4472-afec-197f29eb239d admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:46.507228 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:45:46.720250 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 893/1783] 10.222.0.81 () {60 vars in 1152 bytes} [Wed Jul 20 22:45:46 2022] POST /identity/v3/projects => generated 379 bytes in 484 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:45:46.724724 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c50d14-4120-462a-a84b-7eeeac5be115 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:46.753837 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c50d14-4120-462a-a84b-7eeeac5be115 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:47.134124 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32db7366-cd28-418a-9007-c2bd8e1d1321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:47.139307 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32db7366-cd28-418a-9007-c2bd8e1d1321 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:47.141053 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32db7366-cd28-418a-9007-c2bd8e1d1321 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:47.141354 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32db7366-cd28-418a-9007-c2bd8e1d1321 admin admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:47.144956 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32db7366-cd28-418a-9007-c2bd8e1d1321 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:47.158997 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32db7366-cd28-418a-9007-c2bd8e1d1321 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:47.308949 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 891/1784] 10.222.0.81 () {60 vars in 1143 bytes} [Wed Jul 20 22:45:46 2022] POST /identity/v3/users => generated 296 bytes in 585 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:45:47.313492 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2952fa-936e-4472-afec-197f29eb239d admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:47.329998 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2952fa-936e-4472-afec-197f29eb239d admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:47.341140 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4427e7-9862-4974-8c81-ffdde89250fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:47.347776 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4427e7-9862-4974-8c81-ffdde89250fa admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:47.348616 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4427e7-9862-4974-8c81-ffdde89250fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:47.349241 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4427e7-9862-4974-8c81-ffdde89250fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:47.350244 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4427e7-9862-4974-8c81-ffdde89250fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:47.352409 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4427e7-9862-4974-8c81-ffdde89250fa admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:47.373734 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32db7366-cd28-418a-9007-c2bd8e1d1321 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:47.386735 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 894/1785] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:45:47 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:45:47.399154 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32db7366-cd28-418a-9007-c2bd8e1d1321 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:47.464164 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52f5e45-7960-46cc-8f03-c2b0b7be1c8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:47.479442 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52f5e45-7960-46cc-8f03-c2b0b7be1c8f admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:47.479720 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52f5e45-7960-46cc-8f03-c2b0b7be1c8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:47.485260 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52f5e45-7960-46cc-8f03-c2b0b7be1c8f admin admin] PATH_INFO: `/v3/projects/0d5d606a271547fc88fd98b31e3bf492/users/d5b86d23a1054d59adf072d7eaead23d/roles/d89b6c92d35a401a8517f1d47f6be843` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:47.537424 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52f5e45-7960-46cc-8f03-c2b0b7be1c8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d5d606a271547fc88fd98b31e3bf492, user_id=d5b86d23a1054d59adf072d7eaead23d, role_id=d89b6c92d35a401a8517f1d47f6be843)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:47.540612 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52f5e45-7960-46cc-8f03-c2b0b7be1c8f admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:47.699811 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 892/1786] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:45:47 2022] PUT /identity/v3/projects/0d5d606a271547fc88fd98b31e3bf492/users/d5b86d23a1054d59adf072d7eaead23d/roles/d89b6c92d35a401a8517f1d47f6be843 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:45:47.703185 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4427e7-9862-4974-8c81-ffdde89250fa admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:47.724908 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4427e7-9862-4974-8c81-ffdde89250fa admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:47.781089 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:45:47.995128 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940a9644-5a87-487e-89a5-304c10f780c5 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:47.995423 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940a9644-5a87-487e-89a5-304c10f780c5 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:47.996213 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940a9644-5a87-487e-89a5-304c10f780c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:48.078272 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f686a4e-49dd-498e-8331-6ef5dbdfd43d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:48.079840 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f686a4e-49dd-498e-8331-6ef5dbdfd43d admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:48.080220 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f686a4e-49dd-498e-8331-6ef5dbdfd43d admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:48.080735 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f686a4e-49dd-498e-8331-6ef5dbdfd43d admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:48.081712 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f686a4e-49dd-498e-8331-6ef5dbdfd43d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:48.097357 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f686a4e-49dd-498e-8331-6ef5dbdfd43d admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:48.130456 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 895/1787] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:45:47 2022] GET /identity/v3/roles => generated 1420 bytes in 426 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:45:48.131606 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f686a4e-49dd-498e-8331-6ef5dbdfd43d admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:48.137606 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-9f686a4e-49dd-498e-8331-6ef5dbdfd43d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:48.228231 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-940a9644-5a87-487e-89a5-304c10f780c5 None None] MFA Rules not processed for user `90915e1264324f16bc7bae7e46e0c97b`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:45:48.229601 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-940a9644-5a87-487e-89a5-304c10f780c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:48.395390 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f686a4e-49dd-498e-8331-6ef5dbdfd43d admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:48.407859 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ba3df3-6d21-4e09-84b2-beab1c00365b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecf4db289fba4e4a806198c62d31f7c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:48.408658 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ba3df3-6d21-4e09-84b2-beab1c00365b service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:48.414732 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ba3df3-6d21-4e09-84b2-beab1c00365b service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:48.415349 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ba3df3-6d21-4e09-84b2-beab1c00365b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:48.425733 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-68ba3df3-6d21-4e09-84b2-beab1c00365b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:48.501580 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 893/1788] 10.222.0.81 () {58 vars in 1009 bytes} [Wed Jul 20 22:45:47 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 508 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:45:48.503576 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940a9644-5a87-487e-89a5-304c10f780c5 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:48.517811 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940a9644-5a87-487e-89a5-304c10f780c5 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:48.523573 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d81c0b9-e5fe-4b17-b460-70d5e2f9641c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:48.524905 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d81c0b9-e5fe-4b17-b460-70d5e2f9641c admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:48.525729 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d81c0b9-e5fe-4b17-b460-70d5e2f9641c admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:48.526175 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d81c0b9-e5fe-4b17-b460-70d5e2f9641c admin admin] PATH_INFO: `/v3/projects/0d5d606a271547fc88fd98b31e3bf492/users/d5b86d23a1054d59adf072d7eaead23d/roles/d89b6c92d35a401a8517f1d47f6be843` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:48.530385 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:45:48.531983 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d81c0b9-e5fe-4b17-b460-70d5e2f9641c admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d5d606a271547fc88fd98b31e3bf492, user_id=d5b86d23a1054d59adf072d7eaead23d, role_id=d89b6c92d35a401a8517f1d47f6be843)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:48.537015 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d81c0b9-e5fe-4b17-b460-70d5e2f9641c admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:48.560471 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 894/1789] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:45:48 2022] PUT /identity/v3/projects/0d5d606a271547fc88fd98b31e3bf492/users/d5b86d23a1054d59adf072d7eaead23d/roles/d89b6c92d35a401a8517f1d47f6be843 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:45:48.560471 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb83b53-532d-4095-ac8f-da3e904e25dd None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:48.560471 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb83b53-532d-4095-ac8f-da3e904e25dd None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:48.560471 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb83b53-532d-4095-ac8f-da3e904e25dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:48.594498 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-ccb83b53-532d-4095-ac8f-da3e904e25dd None None] MFA Rules not processed for user `3e072e59bf3e46aaa062ec85b0e2c91a`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:45:48.594627 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ba3df3-6d21-4e09-84b2-beab1c00365b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:48.594722 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-ccb83b53-532d-4095-ac8f-da3e904e25dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:48.595684 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ba3df3-6d21-4e09-84b2-beab1c00365b service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:48.608449 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 896/1790] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:45:48 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 477 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:45:48.610286 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d6e830-1bf0-4fe6-bd1c-1a55add91846 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:48.610619 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d6e830-1bf0-4fe6-bd1c-1a55add91846 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:48.611090 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d6e830-1bf0-4fe6-bd1c-1a55add91846 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:48.670747 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 895/1791] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:45:48 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:45:48.711616 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-b9d6e830-1bf0-4fe6-bd1c-1a55add91846 None None] MFA Rules not processed for user `d5b86d23a1054d59adf072d7eaead23d`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:45:48.722223 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-b9d6e830-1bf0-4fe6-bd1c-1a55add91846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:48.930515 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 897/1792] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:45:48 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 322 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:45:49.023416 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb83b53-532d-4095-ac8f-da3e904e25dd None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:49.025225 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-ccb83b53-532d-4095-ac8f-da3e904e25dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:49.164123 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb83b53-532d-4095-ac8f-da3e904e25dd None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:49.170222 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cdf4b1-52b1-46f7-945c-ad3b374db371 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70e97436a08543a180225561ef8bc817', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:49.171163 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cdf4b1-52b1-46f7-945c-ad3b374db371 service swift] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:49.171506 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cdf4b1-52b1-46f7-945c-ad3b374db371 service swift] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:49.172633 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cdf4b1-52b1-46f7-945c-ad3b374db371 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:49.298461 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cdf4b1-52b1-46f7-945c-ad3b374db371 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:49.300310 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cdf4b1-52b1-46f7-945c-ad3b374db371 service swift] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:49.310683 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 896/1793] 10.222.0.81 () {60 vars in 1384 bytes} [Wed Jul 20 22:45:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 22:45:50.192718 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:45:50.243827 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:45:52.677673 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:45:52.774814 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:45:53.117877 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:45:53.363391 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:45:53.660984 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:45:53.820773 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:DELETE=5,keystone:SELECT=93,keystone:INSERT=2 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:45:53.900681 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:45:54.643489 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:45:55.066313 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:45:55.494665 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:45:55.809889 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d6e830-1bf0-4fe6-bd1c-1a55add91846 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:55.822014 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cdf4b1-52b1-46f7-945c-ad3b374db371 service swift] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:55.896440 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:45:55.903801 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:45:55.933459 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cdf4b1-52b1-46f7-945c-ad3b374db371 service swift] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:55.938472 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449a93e0-0287-441e-a06a-8729379d13b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecf4db289fba4e4a806198c62d31f7c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:55.939219 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449a93e0-0287-441e-a06a-8729379d13b9 service cinder] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:55.939535 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449a93e0-0287-441e-a06a-8729379d13b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:55.939825 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449a93e0-0287-441e-a06a-8729379d13b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:55.948456 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d6e830-1bf0-4fe6-bd1c-1a55add91846 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:55.951552 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449a93e0-0287-441e-a06a-8729379d13b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:55.954174 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449a93e0-0287-441e-a06a-8729379d13b9 service cinder] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:55.963666 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1bba76-b3d2-4f7b-9f24-98c642ddad1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:55.967632 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1bba76-b3d2-4f7b-9f24-98c642ddad1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:55.968387 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1bba76-b3d2-4f7b-9f24-98c642ddad1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:55.968931 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1bba76-b3d2-4f7b-9f24-98c642ddad1c admin admin] PATH_INFO: `/v3/users/531e1e88ccf94f91a058f7b1334836a6` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:55.970542 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1bba76-b3d2-4f7b-9f24-98c642ddad1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=531e1e88ccf94f91a058f7b1334836a6)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:55.972704 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1bba76-b3d2-4f7b-9f24-98c642ddad1c admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:55.977943 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 897/1794] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:45:55 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 22:45:56.015453 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:45:56.059608 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449a93e0-0287-441e-a06a-8729379d13b9 service cinder] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:56.059608 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-449a93e0-0287-441e-a06a-8729379d13b9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:56.074065 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449a93e0-0287-441e-a06a-8729379d13b9 service cinder] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:56.081915 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed508853-5d17-4b24-a521-cf1dafa3261b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:56.083189 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed508853-5d17-4b24-a521-cf1dafa3261b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:56.083535 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed508853-5d17-4b24-a521-cf1dafa3261b admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:56.083920 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed508853-5d17-4b24-a521-cf1dafa3261b admin admin] PATH_INFO: `/v3/users/8b98b2ed85394635a3bed5541c5975d7` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:56.085249 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed508853-5d17-4b24-a521-cf1dafa3261b admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b98b2ed85394635a3bed5541c5975d7)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:56.089017 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed508853-5d17-4b24-a521-cf1dafa3261b admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:56.216219 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-fd1bba76-b3d2-4f7b-9f24-98c642ddad1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '531e1e88ccf94f91a058f7b1334836a6'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:56.217130 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-fd1bba76-b3d2-4f7b-9f24-98c642ddad1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '531e1e88ccf94f91a058f7b1334836a6'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:56.217809 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-fd1bba76-b3d2-4f7b-9f24-98c642ddad1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '531e1e88ccf94f91a058f7b1334836a6'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:56.227174 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-fd1bba76-b3d2-4f7b-9f24-98c642ddad1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '531e1e88ccf94f91a058f7b1334836a6'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:56.336257 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-fd1bba76-b3d2-4f7b-9f24-98c642ddad1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '531e1e88ccf94f91a058f7b1334836a6'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:56.382496 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-ed508853-5d17-4b24-a521-cf1dafa3261b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b98b2ed85394635a3bed5541c5975d7'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:56.386548 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-ed508853-5d17-4b24-a521-cf1dafa3261b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b98b2ed85394635a3bed5541c5975d7'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:56.388119 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-ed508853-5d17-4b24-a521-cf1dafa3261b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b98b2ed85394635a3bed5541c5975d7'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:56.396456 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 898/1795] 10.222.0.81 () {58 vars in 1223 bytes} [Wed Jul 20 22:45:55 2022] DELETE /identity/v3/users/531e1e88ccf94f91a058f7b1334836a6 => generated 0 bytes in 587 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:45:56.404523 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14d25fc-bc26-4b54-a47e-d8490a465e1f None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:56.404935 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14d25fc-bc26-4b54-a47e-d8490a465e1f None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:56.405325 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14d25fc-bc26-4b54-a47e-d8490a465e1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:56.411422 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-ed508853-5d17-4b24-a521-cf1dafa3261b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b98b2ed85394635a3bed5541c5975d7'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:56.438922 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-b14d25fc-bc26-4b54-a47e-d8490a465e1f None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:45:56.441095 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-b14d25fc-bc26-4b54-a47e-d8490a465e1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:56.518605 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-ed508853-5d17-4b24-a521-cf1dafa3261b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b98b2ed85394635a3bed5541c5975d7'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:56.554307 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 899/1796] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:45:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:45:56.556063 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 898/1797] 10.222.0.81 () {58 vars in 1223 bytes} [Wed Jul 20 22:45:56 2022] DELETE /identity/v3/users/8b98b2ed85394635a3bed5541c5975d7 => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:45:56.561521 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5b0d36-bf41-4474-b9e6-a0cf8aef3551 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:56.561820 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5b0d36-bf41-4474-b9e6-a0cf8aef3551 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:56.562100 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5b0d36-bf41-4474-b9e6-a0cf8aef3551 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:56.564384 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed508853-5d17-4b24-a521-cf1dafa3261b admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:56.565905 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-ed508853-5d17-4b24-a521-cf1dafa3261b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:56.595866 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-6e5b0d36-bf41-4474-b9e6-a0cf8aef3551 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:45:56.597014 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-6e5b0d36-bf41-4474-b9e6-a0cf8aef3551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:56.671092 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed508853-5d17-4b24-a521-cf1dafa3261b admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:56.677061 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1594a7-db6f-4a09-9f11-e676b12a4527 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:56.678271 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1594a7-db6f-4a09-9f11-e676b12a4527 service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:56.678639 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1594a7-db6f-4a09-9f11-e676b12a4527 service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:56.678996 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1594a7-db6f-4a09-9f11-e676b12a4527 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:56.685924 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:45:56.711556 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:45:56.721556 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 900/1798] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:45:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 22:45:56.731471 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5b0d36-bf41-4474-b9e6-a0cf8aef3551 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:56.747582 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5b0d36-bf41-4474-b9e6-a0cf8aef3551 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:56.755256 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38bd39c-55c1-41c7-871e-9190aa2103f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:56.756241 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38bd39c-55c1-41c7-871e-9190aa2103f7 service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:56.756630 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38bd39c-55c1-41c7-871e-9190aa2103f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:56.757005 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38bd39c-55c1-41c7-871e-9190aa2103f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:56.775718 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38bd39c-55c1-41c7-871e-9190aa2103f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:56.777180 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38bd39c-55c1-41c7-871e-9190aa2103f7 service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:56.793255 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1594a7-db6f-4a09-9f11-e676b12a4527 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:56.794721 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1594a7-db6f-4a09-9f11-e676b12a4527 service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:56.800050 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 901/1799] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:45:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 22:45:56.820847 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 899/1800] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:45:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:45:57.526577 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38bd39c-55c1-41c7-871e-9190aa2103f7 service neutron] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:57.528273 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-b38bd39c-55c1-41c7-871e-9190aa2103f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:57.638668 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38bd39c-55c1-41c7-871e-9190aa2103f7 service neutron] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:57.645042 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad2f271-6bdb-47c1-b78d-d15779177c51 tempest-VolumeQuotaClassesTest-200629613 tempest-VolumeQuotaClassesTest-200629613-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5b86d23a1054d59adf072d7eaead23d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d5d606a271547fc88fd98b31e3bf492', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:57.645574 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad2f271-6bdb-47c1-b78d-d15779177c51 tempest-VolumeQuotaClassesTest-200629613 tempest-VolumeQuotaClassesTest-200629613-project-admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:57.645869 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad2f271-6bdb-47c1-b78d-d15779177c51 tempest-VolumeQuotaClassesTest-200629613 tempest-VolumeQuotaClassesTest-200629613-project-admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:57.646167 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad2f271-6bdb-47c1-b78d-d15779177c51 tempest-VolumeQuotaClassesTest-200629613 tempest-VolumeQuotaClassesTest-200629613-project-admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:57.646917 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad2f271-6bdb-47c1-b78d-d15779177c51 tempest-VolumeQuotaClassesTest-200629613 tempest-VolumeQuotaClassesTest-200629613-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:45:57.647273 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad2f271-6bdb-47c1-b78d-d15779177c51 tempest-VolumeQuotaClassesTest-200629613 tempest-VolumeQuotaClassesTest-200629613-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:57.648906 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad2f271-6bdb-47c1-b78d-d15779177c51 tempest-VolumeQuotaClassesTest-200629613 tempest-VolumeQuotaClassesTest-200629613-project-admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:57.678999 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 902/1801] 10.222.0.81 () {58 vars in 1152 bytes} [Wed Jul 20 22:45:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:45:57.681316 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1594a7-db6f-4a09-9f11-e676b12a4527 service neutron] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:57.718779 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1594a7-db6f-4a09-9f11-e676b12a4527 service neutron] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:57.749749 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70aa3a5-0ada-4488-b188-bd30638db221 tempest-VolumeQuotaClassesTest-200629613 tempest-VolumeQuotaClassesTest-200629613-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5b86d23a1054d59adf072d7eaead23d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d5d606a271547fc88fd98b31e3bf492', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:57.751900 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70aa3a5-0ada-4488-b188-bd30638db221 tempest-VolumeQuotaClassesTest-200629613 tempest-VolumeQuotaClassesTest-200629613-project-admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:57.752204 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70aa3a5-0ada-4488-b188-bd30638db221 tempest-VolumeQuotaClassesTest-200629613 tempest-VolumeQuotaClassesTest-200629613-project-admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:57.752931 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70aa3a5-0ada-4488-b188-bd30638db221 tempest-VolumeQuotaClassesTest-200629613 tempest-VolumeQuotaClassesTest-200629613-project-admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:57.755177 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70aa3a5-0ada-4488-b188-bd30638db221 tempest-VolumeQuotaClassesTest-200629613 tempest-VolumeQuotaClassesTest-200629613-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:57.759836 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70aa3a5-0ada-4488-b188-bd30638db221 tempest-VolumeQuotaClassesTest-200629613 tempest-VolumeQuotaClassesTest-200629613-project-admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:57.890036 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad2f271-6bdb-47c1-b78d-d15779177c51 tempest-VolumeQuotaClassesTest-200629613 tempest-VolumeQuotaClassesTest-200629613-project-admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:57.892601 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-9ad2f271-6bdb-47c1-b78d-d15779177c51 tempest-VolumeQuotaClassesTest-200629613 tempest-VolumeQuotaClassesTest-200629613-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:57.911732 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 900/1802] 10.222.0.81 () {60 vars in 1152 bytes} [Wed Jul 20 22:45:57 2022] POST /identity/v3/projects => generated 353 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:45:57.912908 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70aa3a5-0ada-4488-b188-bd30638db221 tempest-VolumeQuotaClassesTest-200629613 tempest-VolumeQuotaClassesTest-200629613-project-admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:57.917559 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad2f271-6bdb-47c1-b78d-d15779177c51 tempest-VolumeQuotaClassesTest-200629613 tempest-VolumeQuotaClassesTest-200629613-project-admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:58.060978 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70aa3a5-0ada-4488-b188-bd30638db221 tempest-VolumeQuotaClassesTest-200629613 tempest-VolumeQuotaClassesTest-200629613-project-admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:58.065811 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf19df42-e6c9-4ecd-b680-8472d7a22cb0 tempest-VolumeQuotaClassesTest-200629613 tempest-VolumeQuotaClassesTest-200629613-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5b86d23a1054d59adf072d7eaead23d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d5d606a271547fc88fd98b31e3bf492', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:58.066495 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf19df42-e6c9-4ecd-b680-8472d7a22cb0 tempest-VolumeQuotaClassesTest-200629613 tempest-VolumeQuotaClassesTest-200629613-project-admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:58.066777 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf19df42-e6c9-4ecd-b680-8472d7a22cb0 tempest-VolumeQuotaClassesTest-200629613 tempest-VolumeQuotaClassesTest-200629613-project-admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:58.067095 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf19df42-e6c9-4ecd-b680-8472d7a22cb0 tempest-VolumeQuotaClassesTest-200629613 tempest-VolumeQuotaClassesTest-200629613-project-admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:58.068365 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf19df42-e6c9-4ecd-b680-8472d7a22cb0 tempest-VolumeQuotaClassesTest-200629613 tempest-VolumeQuotaClassesTest-200629613-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:45:58.068709 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf19df42-e6c9-4ecd-b680-8472d7a22cb0 tempest-VolumeQuotaClassesTest-200629613 tempest-VolumeQuotaClassesTest-200629613-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:58.073175 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf19df42-e6c9-4ecd-b680-8472d7a22cb0 tempest-VolumeQuotaClassesTest-200629613 tempest-VolumeQuotaClassesTest-200629613-project-admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:58.086174 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee968c3-bba7-4413-918d-1e91da3b8b48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:58.086730 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee968c3-bba7-4413-918d-1e91da3b8b48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:58.087081 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee968c3-bba7-4413-918d-1e91da3b8b48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:58.087326 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee968c3-bba7-4413-918d-1e91da3b8b48 admin admin] PATH_INFO: `/v3/projects/9ab718882fdc4ec7af24826c3f1a8611` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:58.088543 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee968c3-bba7-4413-918d-1e91da3b8b48 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ab718882fdc4ec7af24826c3f1a8611)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:58.090446 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee968c3-bba7-4413-918d-1e91da3b8b48 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:58.113795 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 901/1803] 10.222.0.81 () {58 vars in 1152 bytes} [Wed Jul 20 22:45:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:45:58.113795 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf19df42-e6c9-4ecd-b680-8472d7a22cb0 tempest-VolumeQuotaClassesTest-200629613 tempest-VolumeQuotaClassesTest-200629613-project-admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:58.125381 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-bf19df42-e6c9-4ecd-b680-8472d7a22cb0 tempest-VolumeQuotaClassesTest-200629613 tempest-VolumeQuotaClassesTest-200629613-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:58.158489 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf19df42-e6c9-4ecd-b680-8472d7a22cb0 tempest-VolumeQuotaClassesTest-200629613 tempest-VolumeQuotaClassesTest-200629613-project-admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:58.174821 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54132a56-0013-4a58-b4b1-9153c039d01a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:58.176261 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54132a56-0013-4a58-b4b1-9153c039d01a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:58.176622 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54132a56-0013-4a58-b4b1-9153c039d01a admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:58.177035 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54132a56-0013-4a58-b4b1-9153c039d01a admin admin] PATH_INFO: `/v3/projects/22824ce78e8a4fc79f8394e536768ab3` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:58.183365 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54132a56-0013-4a58-b4b1-9153c039d01a admin admin] RBAC: Authorizing `identity:delete_project(project_id=22824ce78e8a4fc79f8394e536768ab3)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:58.183365 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54132a56-0013-4a58-b4b1-9153c039d01a admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:58.279836 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-3ee968c3-bba7-4413-918d-1e91da3b8b48 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ab718882fdc4ec7af24826c3f1a8611', 'request_id': 'req-3ee968c3-bba7-4413-918d-1e91da3b8b48'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:58.287855 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-3ee968c3-bba7-4413-918d-1e91da3b8b48 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ab718882fdc4ec7af24826c3f1a8611', 'request_id': 'req-3ee968c3-bba7-4413-918d-1e91da3b8b48'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:58.365797 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-54132a56-0013-4a58-b4b1-9153c039d01a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22824ce78e8a4fc79f8394e536768ab3', 'request_id': 'req-54132a56-0013-4a58-b4b1-9153c039d01a'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:58.369075 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-3ee968c3-bba7-4413-918d-1e91da3b8b48 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ab718882fdc4ec7af24826c3f1a8611', 'request_id': 'req-3ee968c3-bba7-4413-918d-1e91da3b8b48'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:58.398462 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-3ee968c3-bba7-4413-918d-1e91da3b8b48 admin admin] The token cache is being invalidate because project 9ab718882fdc4ec7af24826c3f1a8611 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:45:58.398754 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-3ee968c3-bba7-4413-918d-1e91da3b8b48 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:58.399538 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-3ee968c3-bba7-4413-918d-1e91da3b8b48 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:58.401563 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 903/1804] 10.222.0.81 () {58 vars in 1232 bytes} [Wed Jul 20 22:45:57 2022] DELETE /identity/v3/projects/9ab718882fdc4ec7af24826c3f1a8611 => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:45:58.402449 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee968c3-bba7-4413-918d-1e91da3b8b48 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:58.406396 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-3ee968c3-bba7-4413-918d-1e91da3b8b48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:58.434057 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:45:58.486111 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-54132a56-0013-4a58-b4b1-9153c039d01a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22824ce78e8a4fc79f8394e536768ab3', 'request_id': 'req-54132a56-0013-4a58-b4b1-9153c039d01a'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:58.502283 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-54132a56-0013-4a58-b4b1-9153c039d01a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22824ce78e8a4fc79f8394e536768ab3', 'request_id': 'req-54132a56-0013-4a58-b4b1-9153c039d01a'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:58.528159 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:45:58.529068 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee968c3-bba7-4413-918d-1e91da3b8b48 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:58.537286 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-54132a56-0013-4a58-b4b1-9153c039d01a admin admin] The token cache is being invalidate because project 22824ce78e8a4fc79f8394e536768ab3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:45:58.537286 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-54132a56-0013-4a58-b4b1-9153c039d01a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:58.541144 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-54132a56-0013-4a58-b4b1-9153c039d01a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:58.541144 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 902/1805] 10.222.0.81 () {58 vars in 1232 bytes} [Wed Jul 20 22:45:58 2022] DELETE /identity/v3/projects/22824ce78e8a4fc79f8394e536768ab3 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:45:58.541325 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54132a56-0013-4a58-b4b1-9153c039d01a admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:58.545116 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-54132a56-0013-4a58-b4b1-9153c039d01a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:58.626994 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2def977-cd8f-4638-9817-9fdbe8650582 tempest-VolumeQuotaClassesTest-200629613 tempest-VolumeQuotaClassesTest-200629613-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5b86d23a1054d59adf072d7eaead23d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0d5d606a271547fc88fd98b31e3bf492', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:58.627932 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2def977-cd8f-4638-9817-9fdbe8650582 tempest-VolumeQuotaClassesTest-200629613 tempest-VolumeQuotaClassesTest-200629613-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:58.628225 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2def977-cd8f-4638-9817-9fdbe8650582 tempest-VolumeQuotaClassesTest-200629613 tempest-VolumeQuotaClassesTest-200629613-project-admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:58.628509 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2def977-cd8f-4638-9817-9fdbe8650582 tempest-VolumeQuotaClassesTest-200629613 tempest-VolumeQuotaClassesTest-200629613-project-admin] PATH_INFO: `/v3/projects/7ba8c955141f4570bd4b3fcfdf7d883c` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:58.629895 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2def977-cd8f-4638-9817-9fdbe8650582 tempest-VolumeQuotaClassesTest-200629613 tempest-VolumeQuotaClassesTest-200629613-project-admin] RBAC: Authorizing `identity:delete_project(project_id=7ba8c955141f4570bd4b3fcfdf7d883c)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:58.633379 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2def977-cd8f-4638-9817-9fdbe8650582 tempest-VolumeQuotaClassesTest-200629613 tempest-VolumeQuotaClassesTest-200629613-project-admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:58.671697 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54132a56-0013-4a58-b4b1-9153c039d01a admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:58.687714 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf4d7b2-4d80-4774-a4eb-953983f44cb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:58.688857 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf4d7b2-4d80-4774-a4eb-953983f44cb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:58.689219 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf4d7b2-4d80-4774-a4eb-953983f44cb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:58.689564 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf4d7b2-4d80-4774-a4eb-953983f44cb5 admin admin] PATH_INFO: `/v3/users/ccc26e776c5c46de9dba4dd80b183080` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:58.691188 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf4d7b2-4d80-4774-a4eb-953983f44cb5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ccc26e776c5c46de9dba4dd80b183080)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:58.693521 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf4d7b2-4d80-4774-a4eb-953983f44cb5 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:58.760973 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-b2def977-cd8f-4638-9817-9fdbe8650582 tempest-VolumeQuotaClassesTest-200629613 tempest-VolumeQuotaClassesTest-200629613-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ba8c955141f4570bd4b3fcfdf7d883c', 'request_id': 'req-b2def977-cd8f-4638-9817-9fdbe8650582'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:58.770538 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-b2def977-cd8f-4638-9817-9fdbe8650582 tempest-VolumeQuotaClassesTest-200629613 tempest-VolumeQuotaClassesTest-200629613-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ba8c955141f4570bd4b3fcfdf7d883c', 'request_id': 'req-b2def977-cd8f-4638-9817-9fdbe8650582'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:58.877433 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-b2def977-cd8f-4638-9817-9fdbe8650582 tempest-VolumeQuotaClassesTest-200629613 tempest-VolumeQuotaClassesTest-200629613-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ba8c955141f4570bd4b3fcfdf7d883c', 'request_id': 'req-b2def977-cd8f-4638-9817-9fdbe8650582'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:58.895287 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-b2def977-cd8f-4638-9817-9fdbe8650582 tempest-VolumeQuotaClassesTest-200629613 tempest-VolumeQuotaClassesTest-200629613-project-admin] The token cache is being invalidate because project 7ba8c955141f4570bd4b3fcfdf7d883c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:45:58.895614 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-b2def977-cd8f-4638-9817-9fdbe8650582 tempest-VolumeQuotaClassesTest-200629613 tempest-VolumeQuotaClassesTest-200629613-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:58.896285 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-b2def977-cd8f-4638-9817-9fdbe8650582 tempest-VolumeQuotaClassesTest-200629613 tempest-VolumeQuotaClassesTest-200629613-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:58.915954 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 904/1806] 10.222.0.81 () {58 vars in 1232 bytes} [Wed Jul 20 22:45:58 2022] DELETE /identity/v3/projects/7ba8c955141f4570bd4b3fcfdf7d883c => generated 0 bytes in 496 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:45:58.915954 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408819cc-8f8d-470b-b292-2936a3db29a5 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:58.915954 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408819cc-8f8d-470b-b292-2936a3db29a5 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:58.915954 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408819cc-8f8d-470b-b292-2936a3db29a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:58.937959 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-3cf4d7b2-4d80-4774-a4eb-953983f44cb5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ccc26e776c5c46de9dba4dd80b183080'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:58.938698 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-3cf4d7b2-4d80-4774-a4eb-953983f44cb5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ccc26e776c5c46de9dba4dd80b183080'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:58.939349 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-3cf4d7b2-4d80-4774-a4eb-953983f44cb5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ccc26e776c5c46de9dba4dd80b183080'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:58.955502 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-3cf4d7b2-4d80-4774-a4eb-953983f44cb5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ccc26e776c5c46de9dba4dd80b183080'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:58.968777 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-408819cc-8f8d-470b-b292-2936a3db29a5 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:45:58.970050 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-408819cc-8f8d-470b-b292-2936a3db29a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:59.033103 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:45:59.049046 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-3cf4d7b2-4d80-4774-a4eb-953983f44cb5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ccc26e776c5c46de9dba4dd80b183080'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:59.080054 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 905/1807] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:45:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:45:59.081757 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 903/1808] 10.222.0.81 () {58 vars in 1223 bytes} [Wed Jul 20 22:45:58 2022] DELETE /identity/v3/users/ccc26e776c5c46de9dba4dd80b183080 => generated 0 bytes in 541 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:45:59.083330 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0f40e5-56f9-435e-aacf-97b58877d473 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:59.084409 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0f40e5-56f9-435e-aacf-97b58877d473 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:59.084984 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0f40e5-56f9-435e-aacf-97b58877d473 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:59.126489 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-aa0f40e5-56f9-435e-aacf-97b58877d473 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:45:59.127996 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-aa0f40e5-56f9-435e-aacf-97b58877d473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:59.232616 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 906/1809] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:45:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:45:59.234108 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf4d7b2-4d80-4774-a4eb-953983f44cb5 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:59.245262 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf4d7b2-4d80-4774-a4eb-953983f44cb5 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:59.250901 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dd22e8-b2ea-48ea-a26a-ae5b340c3faf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:59.251658 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dd22e8-b2ea-48ea-a26a-ae5b340c3faf admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:59.251948 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dd22e8-b2ea-48ea-a26a-ae5b340c3faf admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:59.252233 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1dd22e8-b2ea-48ea-a26a-ae5b340c3faf admin admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:59.252991 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dd22e8-b2ea-48ea-a26a-ae5b340c3faf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:45:59.253297 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dd22e8-b2ea-48ea-a26a-ae5b340c3faf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:59.254925 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1dd22e8-b2ea-48ea-a26a-ae5b340c3faf admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:59.274453 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 904/1810] 10.222.0.81 () {58 vars in 1152 bytes} [Wed Jul 20 22:45:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:45:59.276005 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0f40e5-56f9-435e-aacf-97b58877d473 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:59.288800 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0f40e5-56f9-435e-aacf-97b58877d473 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:59.293761 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43018222-ee05-4a00-90b4-aa1c5b50bd33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:59.294543 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43018222-ee05-4a00-90b4-aa1c5b50bd33 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:59.294996 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43018222-ee05-4a00-90b4-aa1c5b50bd33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:59.295289 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43018222-ee05-4a00-90b4-aa1c5b50bd33 admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:59.295947 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43018222-ee05-4a00-90b4-aa1c5b50bd33 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:59.297987 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43018222-ee05-4a00-90b4-aa1c5b50bd33 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:59.469372 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 907/1811] 10.222.0.81 () {60 vars in 1152 bytes} [Wed Jul 20 22:45:59 2022] POST /identity/v3/projects => generated 377 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:45:59.473359 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dd22e8-b2ea-48ea-a26a-ae5b340c3faf admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:59.484809 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1dd22e8-b2ea-48ea-a26a-ae5b340c3faf admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:59.607579 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4635a577-d908-40f5-b581-85297c39494c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:59.607950 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4635a577-d908-40f5-b581-85297c39494c admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:59.608242 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4635a577-d908-40f5-b581-85297c39494c admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:59.608618 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4635a577-d908-40f5-b581-85297c39494c admin admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:59.609358 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4635a577-d908-40f5-b581-85297c39494c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:59.611565 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4635a577-d908-40f5-b581-85297c39494c admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:59.635339 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:45:59.670376 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43018222-ee05-4a00-90b4-aa1c5b50bd33 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:59.673022 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-43018222-ee05-4a00-90b4-aa1c5b50bd33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:59.706307 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43018222-ee05-4a00-90b4-aa1c5b50bd33 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:59.714538 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a5a392-b611-48e3-a88f-d6b97fd16466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:59.715907 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a5a392-b611-48e3-a88f-d6b97fd16466 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:59.716343 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a5a392-b611-48e3-a88f-d6b97fd16466 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:59.716840 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a5a392-b611-48e3-a88f-d6b97fd16466 admin admin] PATH_INFO: `/v3/projects/9bab059a851f4964b3eb7c40e6e9c84a` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:59.718260 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a5a392-b611-48e3-a88f-d6b97fd16466 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9bab059a851f4964b3eb7c40e6e9c84a)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:59.720925 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a5a392-b611-48e3-a88f-d6b97fd16466 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:59.743970 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 905/1812] 10.222.0.81 () {60 vars in 1143 bytes} [Wed Jul 20 22:45:59 2022] POST /identity/v3/users => generated 289 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:45:59.748124 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4635a577-d908-40f5-b581-85297c39494c admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:59.763830 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4635a577-d908-40f5-b581-85297c39494c admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:59.778029 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10be0fd-69a3-4269-81a4-ca14c7c93046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:59.779188 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10be0fd-69a3-4269-81a4-ca14c7c93046 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:59.780849 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10be0fd-69a3-4269-81a4-ca14c7c93046 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:59.781421 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10be0fd-69a3-4269-81a4-ca14c7c93046 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:59.782479 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10be0fd-69a3-4269-81a4-ca14c7c93046 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:59.784904 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10be0fd-69a3-4269-81a4-ca14c7c93046 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:59.799411 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 906/1813] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:45:59 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 22:45:59.800589 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10be0fd-69a3-4269-81a4-ca14c7c93046 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:45:59.802729 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-c10be0fd-69a3-4269-81a4-ca14c7c93046 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:45:59.815990 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10be0fd-69a3-4269-81a4-ca14c7c93046 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:45:59.821329 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14b1c9e-13a7-4517-ad45-1a2ebb7cea0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:45:59.822691 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14b1c9e-13a7-4517-ad45-1a2ebb7cea0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:45:59.823163 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14b1c9e-13a7-4517-ad45-1a2ebb7cea0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:45:59.823589 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14b1c9e-13a7-4517-ad45-1a2ebb7cea0b admin admin] PATH_INFO: `/v3/users/903fa5c85a8047c1ad504413b77e4f28` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:45:59.825146 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14b1c9e-13a7-4517-ad45-1a2ebb7cea0b admin admin] RBAC: Authorizing `identity:delete_user(user_id=903fa5c85a8047c1ad504413b77e4f28)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:45:59.827441 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14b1c9e-13a7-4517-ad45-1a2ebb7cea0b admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:45:59.883949 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-08a5a392-b611-48e3-a88f-d6b97fd16466 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9bab059a851f4964b3eb7c40e6e9c84a', 'request_id': 'req-08a5a392-b611-48e3-a88f-d6b97fd16466'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:59.895786 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-08a5a392-b611-48e3-a88f-d6b97fd16466 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9bab059a851f4964b3eb7c40e6e9c84a', 'request_id': 'req-08a5a392-b611-48e3-a88f-d6b97fd16466'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:45:59.982758 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-08a5a392-b611-48e3-a88f-d6b97fd16466 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9bab059a851f4964b3eb7c40e6e9c84a', 'request_id': 'req-08a5a392-b611-48e3-a88f-d6b97fd16466'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:00.001152 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-08a5a392-b611-48e3-a88f-d6b97fd16466 admin admin] The token cache is being invalidate because project 9bab059a851f4964b3eb7c40e6e9c84a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:46:00.001721 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-08a5a392-b611-48e3-a88f-d6b97fd16466 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:00.003184 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-08a5a392-b611-48e3-a88f-d6b97fd16466 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:00.005966 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 908/1814] 10.222.0.81 () {58 vars in 1232 bytes} [Wed Jul 20 22:45:59 2022] DELETE /identity/v3/projects/9bab059a851f4964b3eb7c40e6e9c84a => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 22:46:00.007728 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a5a392-b611-48e3-a88f-d6b97fd16466 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:00.010109 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-08a5a392-b611-48e3-a88f-d6b97fd16466 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:46:00.088301 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:46:00.118960 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a5a392-b611-48e3-a88f-d6b97fd16466 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:00.120552 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-a14b1c9e-13a7-4517-ad45-1a2ebb7cea0b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '903fa5c85a8047c1ad504413b77e4f28'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:00.121334 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-a14b1c9e-13a7-4517-ad45-1a2ebb7cea0b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '903fa5c85a8047c1ad504413b77e4f28'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:00.122441 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-a14b1c9e-13a7-4517-ad45-1a2ebb7cea0b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '903fa5c85a8047c1ad504413b77e4f28'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:00.137025 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806b73bc-19c2-425d-9f25-434f3f6c4409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:00.137025 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806b73bc-19c2-425d-9f25-434f3f6c4409 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:00.137025 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806b73bc-19c2-425d-9f25-434f3f6c4409 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:00.137025 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806b73bc-19c2-425d-9f25-434f3f6c4409 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:00.138038 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806b73bc-19c2-425d-9f25-434f3f6c4409 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:00.138284 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-a14b1c9e-13a7-4517-ad45-1a2ebb7cea0b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '903fa5c85a8047c1ad504413b77e4f28'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:00.138367 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806b73bc-19c2-425d-9f25-434f3f6c4409 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:00.150250 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 909/1815] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:46:00 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:46:00.150250 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806b73bc-19c2-425d-9f25-434f3f6c4409 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:00.151416 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-806b73bc-19c2-425d-9f25-434f3f6c4409 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:46:00.166957 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806b73bc-19c2-425d-9f25-434f3f6c4409 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:00.172142 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a38052-2c0a-4aa8-8524-869881d9fc28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:00.173834 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a38052-2c0a-4aa8-8524-869881d9fc28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:00.173834 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a38052-2c0a-4aa8-8524-869881d9fc28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:00.173834 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a38052-2c0a-4aa8-8524-869881d9fc28 admin admin] PATH_INFO: `/v3/users/a6586620f2924b2c93d723af74cb3fac` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:00.175453 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a38052-2c0a-4aa8-8524-869881d9fc28 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6586620f2924b2c93d723af74cb3fac)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:00.177465 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a38052-2c0a-4aa8-8524-869881d9fc28 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:00.220681 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-a14b1c9e-13a7-4517-ad45-1a2ebb7cea0b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '903fa5c85a8047c1ad504413b77e4f28'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:00.251437 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 907/1816] 10.222.0.81 () {58 vars in 1223 bytes} [Wed Jul 20 22:45:59 2022] DELETE /identity/v3/users/903fa5c85a8047c1ad504413b77e4f28 => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:46:00.252143 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14b1c9e-13a7-4517-ad45-1a2ebb7cea0b admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:00.255359 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-a14b1c9e-13a7-4517-ad45-1a2ebb7cea0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:46:00.339970 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14b1c9e-13a7-4517-ad45-1a2ebb7cea0b admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:00.345230 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf0865d-0fc4-43fa-9084-f738a2ae8900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:00.346149 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf0865d-0fc4-43fa-9084-f738a2ae8900 admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:00.346438 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf0865d-0fc4-43fa-9084-f738a2ae8900 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:00.346724 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf0865d-0fc4-43fa-9084-f738a2ae8900 admin admin] PATH_INFO: `/v3/projects/2051ea10919f4922a6b237dbc996b6d1/users/a21e40cfdac143088ed1c32082f51beb/roles/559942c4de1744169f8b5e0404ecde6b` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:00.364550 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf0865d-0fc4-43fa-9084-f738a2ae8900 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2051ea10919f4922a6b237dbc996b6d1, user_id=a21e40cfdac143088ed1c32082f51beb, role_id=559942c4de1744169f8b5e0404ecde6b)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:00.367272 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf0865d-0fc4-43fa-9084-f738a2ae8900 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:00.397725 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-f1a38052-2c0a-4aa8-8524-869881d9fc28 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6586620f2924b2c93d723af74cb3fac'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:00.398427 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-f1a38052-2c0a-4aa8-8524-869881d9fc28 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6586620f2924b2c93d723af74cb3fac'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:00.399357 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-f1a38052-2c0a-4aa8-8524-869881d9fc28 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6586620f2924b2c93d723af74cb3fac'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:00.409608 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-f1a38052-2c0a-4aa8-8524-869881d9fc28 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6586620f2924b2c93d723af74cb3fac'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:00.472487 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 908/1817] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:46:00 2022] PUT /identity/v3/projects/2051ea10919f4922a6b237dbc996b6d1/users/a21e40cfdac143088ed1c32082f51beb/roles/559942c4de1744169f8b5e0404ecde6b => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:46:00.476617 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1596ac87-eb1d-4ac7-8eeb-f8a0349a5b42 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:00.476941 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1596ac87-eb1d-4ac7-8eeb-f8a0349a5b42 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:00.477227 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1596ac87-eb1d-4ac7-8eeb-f8a0349a5b42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:00.496725 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-1596ac87-eb1d-4ac7-8eeb-f8a0349a5b42 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:46:00.498009 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-1596ac87-eb1d-4ac7-8eeb-f8a0349a5b42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:46:00.522641 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-f1a38052-2c0a-4aa8-8524-869881d9fc28 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6586620f2924b2c93d723af74cb3fac'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:00.538181 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:INSERT=3,keystone:SELECT=87,keystone:DELETE=10 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:46:00.562575 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 910/1818] 10.222.0.81 () {58 vars in 1223 bytes} [Wed Jul 20 22:46:00 2022] DELETE /identity/v3/users/a6586620f2924b2c93d723af74cb3fac => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:46:00.564185 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e328efb-a8e3-4e41-acbc-f18eef423ad8 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:00.564459 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e328efb-a8e3-4e41-acbc-f18eef423ad8 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:00.564791 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e328efb-a8e3-4e41-acbc-f18eef423ad8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:00.580937 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 909/1819] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:46:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:46:00.607778 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-8e328efb-a8e3-4e41-acbc-f18eef423ad8 None None] MFA Rules not processed for user `a21e40cfdac143088ed1c32082f51beb`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:46:00.608299 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1596ac87-eb1d-4ac7-8eeb-f8a0349a5b42 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:00.610121 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-8e328efb-a8e3-4e41-acbc-f18eef423ad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:46:00.610256 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-1596ac87-eb1d-4ac7-8eeb-f8a0349a5b42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:46:00.681223 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1596ac87-eb1d-4ac7-8eeb-f8a0349a5b42 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:00.691631 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 911/1820] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:46:00 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:46:00.695955 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e328efb-a8e3-4e41-acbc-f18eef423ad8 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:00.697487 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-8e328efb-a8e3-4e41-acbc-f18eef423ad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:46:00.745808 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b8e72c-c8db-417a-ac7d-b0797bfdcf5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:00.756205 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b8e72c-c8db-417a-ac7d-b0797bfdcf5b service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:00.756205 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b8e72c-c8db-417a-ac7d-b0797bfdcf5b service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:00.756205 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b8e72c-c8db-417a-ac7d-b0797bfdcf5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:00.756496 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:46:00.790985 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e328efb-a8e3-4e41-acbc-f18eef423ad8 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:00.796129 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3290e1-c247-4006-baa1-145161f97160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:00.797107 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3290e1-c247-4006-baa1-145161f97160 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:00.797411 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3290e1-c247-4006-baa1-145161f97160 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:00.797764 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3290e1-c247-4006-baa1-145161f97160 admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:00.798622 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3290e1-c247-4006-baa1-145161f97160 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:00.800700 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3290e1-c247-4006-baa1-145161f97160 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:00.819475 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b8e72c-c8db-417a-ac7d-b0797bfdcf5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:00.821114 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b8e72c-c8db-417a-ac7d-b0797bfdcf5b service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:00.831338 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 910/1821] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:46:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 22:46:01.125105 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 912/1822] 10.222.0.81 () {60 vars in 1152 bytes} [Wed Jul 20 22:46:00 2022] POST /identity/v3/projects => generated 375 bytes in 428 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:46:01.128226 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b8e72c-c8db-417a-ac7d-b0797bfdcf5b service neutron] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:01.143707 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b8e72c-c8db-417a-ac7d-b0797bfdcf5b service neutron] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:01.254600 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1506297e-f85c-421e-b5f8-6f44ada17ba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:01.256993 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1506297e-f85c-421e-b5f8-6f44ada17ba5 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:01.257295 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1506297e-f85c-421e-b5f8-6f44ada17ba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:01.257580 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1506297e-f85c-421e-b5f8-6f44ada17ba5 admin admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:01.259164 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1506297e-f85c-421e-b5f8-6f44ada17ba5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:01.261398 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1506297e-f85c-421e-b5f8-6f44ada17ba5 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:01.284010 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:46:01.352899 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 911/1823] 10.222.0.81 () {60 vars in 1143 bytes} [Wed Jul 20 22:46:01 2022] POST /identity/v3/users => generated 294 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 22:46:01.356593 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3290e1-c247-4006-baa1-145161f97160 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:01.373771 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3290e1-c247-4006-baa1-145161f97160 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:01.379903 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d16c9b-d8bc-4e7b-acd1-cc0e7d26a961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:01.380837 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d16c9b-d8bc-4e7b-acd1-cc0e7d26a961 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:01.381331 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d16c9b-d8bc-4e7b-acd1-cc0e7d26a961 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:01.382214 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d16c9b-d8bc-4e7b-acd1-cc0e7d26a961 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:01.383268 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d16c9b-d8bc-4e7b-acd1-cc0e7d26a961 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:01.385179 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d16c9b-d8bc-4e7b-acd1-cc0e7d26a961 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:01.403533 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 913/1824] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:46:01 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:46:01.408225 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1506297e-f85c-421e-b5f8-6f44ada17ba5 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:01.422799 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1506297e-f85c-421e-b5f8-6f44ada17ba5 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:01.428132 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d16c9b-d8bc-4e7b-acd1-cc0e7d26a961 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:01.430230 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-16d16c9b-d8bc-4e7b-acd1-cc0e7d26a961 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:46:01.436684 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af8fd0d-d8e0-4877-bf43-a3e5057fe52b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:01.440224 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af8fd0d-d8e0-4877-bf43-a3e5057fe52b admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:01.440563 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af8fd0d-d8e0-4877-bf43-a3e5057fe52b admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:01.440859 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af8fd0d-d8e0-4877-bf43-a3e5057fe52b admin admin] PATH_INFO: `/v3/projects/9e6f824af4f34890b940123f65fe7aad/users/2e55d75a94e84119bc95f33f791af08d/roles/d89b6c92d35a401a8517f1d47f6be843` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:01.451866 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d16c9b-d8bc-4e7b-acd1-cc0e7d26a961 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:01.456976 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea664da-fed1-446d-ac53-6fd6d1223447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:01.458121 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea664da-fed1-446d-ac53-6fd6d1223447 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:01.459131 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea664da-fed1-446d-ac53-6fd6d1223447 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:01.459131 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea664da-fed1-446d-ac53-6fd6d1223447 admin admin] PATH_INFO: `/v3/projects/8b5b30e1cf174e0790e40c303ff7ba46` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:01.460819 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea664da-fed1-446d-ac53-6fd6d1223447 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b5b30e1cf174e0790e40c303ff7ba46)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:01.463170 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea664da-fed1-446d-ac53-6fd6d1223447 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:01.480574 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af8fd0d-d8e0-4877-bf43-a3e5057fe52b admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e6f824af4f34890b940123f65fe7aad, user_id=2e55d75a94e84119bc95f33f791af08d, role_id=d89b6c92d35a401a8517f1d47f6be843)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:01.487314 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af8fd0d-d8e0-4877-bf43-a3e5057fe52b admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:01.615993 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 912/1825] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:46:01 2022] PUT /identity/v3/projects/9e6f824af4f34890b940123f65fe7aad/users/2e55d75a94e84119bc95f33f791af08d/roles/d89b6c92d35a401a8517f1d47f6be843 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:46:01.620200 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af8fd0d-d8e0-4877-bf43-a3e5057fe52b admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:01.625709 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-5af8fd0d-d8e0-4877-bf43-a3e5057fe52b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:46:01.636475 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-7ea664da-fed1-446d-ac53-6fd6d1223447 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b5b30e1cf174e0790e40c303ff7ba46', 'request_id': 'req-7ea664da-fed1-446d-ac53-6fd6d1223447'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:01.645647 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-7ea664da-fed1-446d-ac53-6fd6d1223447 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b5b30e1cf174e0790e40c303ff7ba46', 'request_id': 'req-7ea664da-fed1-446d-ac53-6fd6d1223447'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:01.728659 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-7ea664da-fed1-446d-ac53-6fd6d1223447 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b5b30e1cf174e0790e40c303ff7ba46', 'request_id': 'req-7ea664da-fed1-446d-ac53-6fd6d1223447'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:01.737941 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af8fd0d-d8e0-4877-bf43-a3e5057fe52b admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:01.745946 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-7ea664da-fed1-446d-ac53-6fd6d1223447 admin admin] The token cache is being invalidate because project 8b5b30e1cf174e0790e40c303ff7ba46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:46:01.747523 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-7ea664da-fed1-446d-ac53-6fd6d1223447 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:01.750279 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-7ea664da-fed1-446d-ac53-6fd6d1223447 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:01.758726 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 914/1826] 10.222.0.81 () {58 vars in 1232 bytes} [Wed Jul 20 22:46:01 2022] DELETE /identity/v3/projects/8b5b30e1cf174e0790e40c303ff7ba46 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:46:01.760476 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea664da-fed1-446d-ac53-6fd6d1223447 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:01.763710 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-7ea664da-fed1-446d-ac53-6fd6d1223447 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:46:01.801057 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d98d43-c19d-453b-9218-7660f41a6cba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:01.803879 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d98d43-c19d-453b-9218-7660f41a6cba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:01.803879 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d98d43-c19d-453b-9218-7660f41a6cba admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:01.803879 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d98d43-c19d-453b-9218-7660f41a6cba admin admin] PATH_INFO: `/v3/projects/44d624a90e2a4e4196a0ebdd778b566e` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:01.803879 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d98d43-c19d-453b-9218-7660f41a6cba admin admin] RBAC: Authorizing `identity:delete_project(project_id=44d624a90e2a4e4196a0ebdd778b566e)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:01.805807 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d98d43-c19d-453b-9218-7660f41a6cba admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:01.847064 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea664da-fed1-446d-ac53-6fd6d1223447 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:01.852458 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9da40cc-49ad-42dd-817e-aa069f9cb5a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:01.853240 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9da40cc-49ad-42dd-817e-aa069f9cb5a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:01.853531 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9da40cc-49ad-42dd-817e-aa069f9cb5a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:01.853817 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9da40cc-49ad-42dd-817e-aa069f9cb5a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:01.854561 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9da40cc-49ad-42dd-817e-aa069f9cb5a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:01.856290 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9da40cc-49ad-42dd-817e-aa069f9cb5a8 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:01.871786 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 915/1827] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:46:01 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:46:01.871786 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5238f0e3-2961-4698-83d7-5e829d3a4a35 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:01.871786 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5238f0e3-2961-4698-83d7-5e829d3a4a35 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:01.871786 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5238f0e3-2961-4698-83d7-5e829d3a4a35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:01.891856 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-5238f0e3-2961-4698-83d7-5e829d3a4a35 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:46:01.893119 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-5238f0e3-2961-4698-83d7-5e829d3a4a35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:46:01.901444 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 916/1828] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:46:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:46:01.902280 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5238f0e3-2961-4698-83d7-5e829d3a4a35 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:01.912851 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5238f0e3-2961-4698-83d7-5e829d3a4a35 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:01.918349 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63880999-aec6-4b74-bbcb-03850683901f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:01.919472 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63880999-aec6-4b74-bbcb-03850683901f admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:01.919763 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63880999-aec6-4b74-bbcb-03850683901f admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:01.920048 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63880999-aec6-4b74-bbcb-03850683901f admin admin] PATH_INFO: `/v3/projects/9e6f824af4f34890b940123f65fe7aad/users/2e55d75a94e84119bc95f33f791af08d/roles/d89b6c92d35a401a8517f1d47f6be843` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:01.923597 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63880999-aec6-4b74-bbcb-03850683901f admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e6f824af4f34890b940123f65fe7aad, user_id=2e55d75a94e84119bc95f33f791af08d, role_id=d89b6c92d35a401a8517f1d47f6be843)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:01.926287 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63880999-aec6-4b74-bbcb-03850683901f admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:01.938260 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 917/1829] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:46:01 2022] PUT /identity/v3/projects/9e6f824af4f34890b940123f65fe7aad/users/2e55d75a94e84119bc95f33f791af08d/roles/d89b6c92d35a401a8517f1d47f6be843 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:46:01.939638 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8437fd29-91a7-43a1-9df3-2c83e51b19b8 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:01.939930 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8437fd29-91a7-43a1-9df3-2c83e51b19b8 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:01.940213 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8437fd29-91a7-43a1-9df3-2c83e51b19b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:01.949596 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-22d98d43-c19d-453b-9218-7660f41a6cba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44d624a90e2a4e4196a0ebdd778b566e', 'request_id': 'req-22d98d43-c19d-453b-9218-7660f41a6cba'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:01.960659 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-8437fd29-91a7-43a1-9df3-2c83e51b19b8 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:46:01.961989 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-8437fd29-91a7-43a1-9df3-2c83e51b19b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:46:02.023338 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 918/1830] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:46:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:46:02.024703 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4080803f-8a6f-4ab1-996b-8ccd75c9ca9d None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:02.024992 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4080803f-8a6f-4ab1-996b-8ccd75c9ca9d None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:02.025276 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4080803f-8a6f-4ab1-996b-8ccd75c9ca9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:02.050094 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-22d98d43-c19d-453b-9218-7660f41a6cba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44d624a90e2a4e4196a0ebdd778b566e', 'request_id': 'req-22d98d43-c19d-453b-9218-7660f41a6cba'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:02.053153 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=89,keystone:INSERT=5,keystone:DELETE=6 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:46:02.053623 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-4080803f-8a6f-4ab1-996b-8ccd75c9ca9d None None] MFA Rules not processed for user `2e55d75a94e84119bc95f33f791af08d`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:46:02.054876 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-4080803f-8a6f-4ab1-996b-8ccd75c9ca9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:46:02.057983 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-22d98d43-c19d-453b-9218-7660f41a6cba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44d624a90e2a4e4196a0ebdd778b566e', 'request_id': 'req-22d98d43-c19d-453b-9218-7660f41a6cba'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:02.073547 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-22d98d43-c19d-453b-9218-7660f41a6cba admin admin] The token cache is being invalidate because project 44d624a90e2a4e4196a0ebdd778b566e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:46:02.074265 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-22d98d43-c19d-453b-9218-7660f41a6cba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:02.076469 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-22d98d43-c19d-453b-9218-7660f41a6cba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:02.079935 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 913/1831] 10.222.0.81 () {58 vars in 1232 bytes} [Wed Jul 20 22:46:01 2022] DELETE /identity/v3/projects/44d624a90e2a4e4196a0ebdd778b566e => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:46:02.082921 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d98d43-c19d-453b-9218-7660f41a6cba admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:02.083357 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-22d98d43-c19d-453b-9218-7660f41a6cba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:46:02.135362 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:46:02.147165 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 919/1832] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:46:02 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:46:02.148072 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4080803f-8a6f-4ab1-996b-8ccd75c9ca9d None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:02.149398 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-4080803f-8a6f-4ab1-996b-8ccd75c9ca9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:46:02.164141 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4080803f-8a6f-4ab1-996b-8ccd75c9ca9d None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:02.165908 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d98d43-c19d-453b-9218-7660f41a6cba admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:02.169633 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ddb8f4-da4c-4ccd-acc3-2e65127efd81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:02.170568 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ddb8f4-da4c-4ccd-acc3-2e65127efd81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:02.174572 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ddb8f4-da4c-4ccd-acc3-2e65127efd81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:02.174572 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ddb8f4-da4c-4ccd-acc3-2e65127efd81 admin admin] PATH_INFO: `/v3/users/d5b86d23a1054d59adf072d7eaead23d` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:02.174775 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12f8ce2-1018-47af-a3c5-a5f294675d60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:02.174775 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12f8ce2-1018-47af-a3c5-a5f294675d60 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:02.174775 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12f8ce2-1018-47af-a3c5-a5f294675d60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:02.175215 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ddb8f4-da4c-4ccd-acc3-2e65127efd81 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5b86d23a1054d59adf072d7eaead23d)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:02.175318 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12f8ce2-1018-47af-a3c5-a5f294675d60 admin admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:02.175318 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12f8ce2-1018-47af-a3c5-a5f294675d60 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:46:02.175318 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12f8ce2-1018-47af-a3c5-a5f294675d60 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:02.175501 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ddb8f4-da4c-4ccd-acc3-2e65127efd81 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:02.176486 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12f8ce2-1018-47af-a3c5-a5f294675d60 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:02.199066 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 914/1833] 10.222.0.81 () {58 vars in 1152 bytes} [Wed Jul 20 22:46:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:46:02.199066 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12f8ce2-1018-47af-a3c5-a5f294675d60 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:02.204182 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-d12f8ce2-1018-47af-a3c5-a5f294675d60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:46:02.264331 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12f8ce2-1018-47af-a3c5-a5f294675d60 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:02.268014 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b8e814-df1e-4a29-bbe8-a4dc139f8cc8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecf4db289fba4e4a806198c62d31f7c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:02.268678 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b8e814-df1e-4a29-bbe8-a4dc139f8cc8 service cinder] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:02.268971 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b8e814-df1e-4a29-bbe8-a4dc139f8cc8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:02.269255 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b8e814-df1e-4a29-bbe8-a4dc139f8cc8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:02.271471 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-f0b8e814-df1e-4a29-bbe8-a4dc139f8cc8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:46:02.336571 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b8e814-df1e-4a29-bbe8-a4dc139f8cc8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:02.338019 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b8e814-df1e-4a29-bbe8-a4dc139f8cc8 service cinder] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:02.353258 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 915/1834] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:46:02 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:46:02.353936 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b8e814-df1e-4a29-bbe8-a4dc139f8cc8 service cinder] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:02.365469 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b8e814-df1e-4a29-bbe8-a4dc139f8cc8 service cinder] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:02.370229 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-b1ddb8f4-da4c-4ccd-acc3-2e65127efd81 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5b86d23a1054d59adf072d7eaead23d'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:02.370573 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a77ce7-5fd1-489f-84dd-ff1ef49a67cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:02.371113 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-b1ddb8f4-da4c-4ccd-acc3-2e65127efd81 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5b86d23a1054d59adf072d7eaead23d'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:02.371282 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a77ce7-5fd1-489f-84dd-ff1ef49a67cf admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:02.371576 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a77ce7-5fd1-489f-84dd-ff1ef49a67cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:02.371863 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a77ce7-5fd1-489f-84dd-ff1ef49a67cf admin admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:02.372324 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-b1ddb8f4-da4c-4ccd-acc3-2e65127efd81 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5b86d23a1054d59adf072d7eaead23d'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:02.372561 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a77ce7-5fd1-489f-84dd-ff1ef49a67cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:02.374576 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a77ce7-5fd1-489f-84dd-ff1ef49a67cf admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:02.381022 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-b1ddb8f4-da4c-4ccd-acc3-2e65127efd81 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5b86d23a1054d59adf072d7eaead23d'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:02.524633 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-b1ddb8f4-da4c-4ccd-acc3-2e65127efd81 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5b86d23a1054d59adf072d7eaead23d'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:02.537225 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 916/1835] 10.222.0.81 () {60 vars in 1152 bytes} [Wed Jul 20 22:46:02 2022] POST /identity/v3/projects => generated 381 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:46:02.543492 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a77ce7-5fd1-489f-84dd-ff1ef49a67cf admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:02.548190 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-93a77ce7-5fd1-489f-84dd-ff1ef49a67cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:46:02.572658 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 920/1836] 10.222.0.81 () {58 vars in 1223 bytes} [Wed Jul 20 22:46:02 2022] DELETE /identity/v3/users/d5b86d23a1054d59adf072d7eaead23d => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:46:02.667904 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a77ce7-5fd1-489f-84dd-ff1ef49a67cf admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:02.741762 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fd32fa-1aaf-43c5-bde4-dfa831106b66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:02.742631 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fd32fa-1aaf-43c5-bde4-dfa831106b66 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:02.743101 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fd32fa-1aaf-43c5-bde4-dfa831106b66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:02.743827 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fd32fa-1aaf-43c5-bde4-dfa831106b66 admin admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:02.745112 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fd32fa-1aaf-43c5-bde4-dfa831106b66 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:02.747566 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fd32fa-1aaf-43c5-bde4-dfa831106b66 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:02.857765 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 917/1837] 10.222.0.81 () {60 vars in 1143 bytes} [Wed Jul 20 22:46:02 2022] POST /identity/v3/users => generated 291 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:46:02.862536 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ddb8f4-da4c-4ccd-acc3-2e65127efd81 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:02.876685 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ddb8f4-da4c-4ccd-acc3-2e65127efd81 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:02.884536 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efd609f-7c9b-4fa2-96a5-ed6946362095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:02.885229 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efd609f-7c9b-4fa2-96a5-ed6946362095 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:02.885521 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efd609f-7c9b-4fa2-96a5-ed6946362095 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:02.885800 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efd609f-7c9b-4fa2-96a5-ed6946362095 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:02.886534 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efd609f-7c9b-4fa2-96a5-ed6946362095 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:02.888201 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efd609f-7c9b-4fa2-96a5-ed6946362095 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:02.899070 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 921/1838] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:46:02 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:46:02.903126 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fd32fa-1aaf-43c5-bde4-dfa831106b66 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:02.915820 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fd32fa-1aaf-43c5-bde4-dfa831106b66 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:02.922282 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b99055-f97d-4c5f-9a13-72cbe9ee4a74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:02.923258 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b99055-f97d-4c5f-9a13-72cbe9ee4a74 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:02.923606 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b99055-f97d-4c5f-9a13-72cbe9ee4a74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:02.923950 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b99055-f97d-4c5f-9a13-72cbe9ee4a74 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:02.924749 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b99055-f97d-4c5f-9a13-72cbe9ee4a74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:02.926352 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b99055-f97d-4c5f-9a13-72cbe9ee4a74 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:02.936912 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 918/1839] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:46:02 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:46:02.941224 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efd609f-7c9b-4fa2-96a5-ed6946362095 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:02.948754 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced54d74-a7ed-4163-9a58-b208c28a74b8 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:02.948754 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced54d74-a7ed-4163-9a58-b208c28a74b8 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:02.948754 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced54d74-a7ed-4163-9a58-b208c28a74b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:02.951850 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efd609f-7c9b-4fa2-96a5-ed6946362095 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:02.959916 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9005c7b-db5d-4610-99e4-edfe9ce1bba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:02.961052 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9005c7b-db5d-4610-99e4-edfe9ce1bba0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:02.961463 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9005c7b-db5d-4610-99e4-edfe9ce1bba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:02.961843 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9005c7b-db5d-4610-99e4-edfe9ce1bba0 admin admin] PATH_INFO: `/v3/projects/edc43b27235c403098ea4232fc152c8c/users/62e39f0e73c14dc0928445355f277de3/roles/559942c4de1744169f8b5e0404ecde6b` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:02.965413 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-ced54d74-a7ed-4163-9a58-b208c28a74b8 None None] MFA Rules not processed for user `333c2b1b11cc4c478abfd965a72e9d66`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:46:02.966672 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-ced54d74-a7ed-4163-9a58-b208c28a74b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:46:02.991266 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9005c7b-db5d-4610-99e4-edfe9ce1bba0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=edc43b27235c403098ea4232fc152c8c, user_id=62e39f0e73c14dc0928445355f277de3, role_id=559942c4de1744169f8b5e0404ecde6b)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:02.994001 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9005c7b-db5d-4610-99e4-edfe9ce1bba0 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:03.043764 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 919/1840] 10.222.0.23 () {58 vars in 1015 bytes} [Wed Jul 20 22:46:02 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 22:46:03.112529 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced54d74-a7ed-4163-9a58-b208c28a74b8 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:03.114102 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-ced54d74-a7ed-4163-9a58-b208c28a74b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:46:03.125828 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced54d74-a7ed-4163-9a58-b208c28a74b8 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:03.129788 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f070468-b583-4196-9f0d-73714f68d2b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:03.130444 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f070468-b583-4196-9f0d-73714f68d2b3 service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:03.130729 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f070468-b583-4196-9f0d-73714f68d2b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:03.131023 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f070468-b583-4196-9f0d-73714f68d2b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:03.161323 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 922/1841] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:46:02 2022] PUT /identity/v3/projects/edc43b27235c403098ea4232fc152c8c/users/62e39f0e73c14dc0928445355f277de3/roles/559942c4de1744169f8b5e0404ecde6b => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:46:03.162330 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9005c7b-db5d-4610-99e4-edfe9ce1bba0 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:03.163973 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-e9005c7b-db5d-4610-99e4-edfe9ce1bba0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:46:03.175417 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f070468-b583-4196-9f0d-73714f68d2b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:03.176864 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f070468-b583-4196-9f0d-73714f68d2b3 service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:03.201816 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 920/1842] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:46:03 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 22:46:03.202660 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f070468-b583-4196-9f0d-73714f68d2b3 service neutron] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:03.204931 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-0f070468-b583-4196-9f0d-73714f68d2b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:46:03.226472 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f070468-b583-4196-9f0d-73714f68d2b3 service neutron] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:03.264524 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9005c7b-db5d-4610-99e4-edfe9ce1bba0 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:03.270010 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad37ca3-afcc-4cb7-b3e4-4703356b0643 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:03.270562 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad37ca3-afcc-4cb7-b3e4-4703356b0643 service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:03.271934 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad37ca3-afcc-4cb7-b3e4-4703356b0643 service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:03.272296 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad37ca3-afcc-4cb7-b3e4-4703356b0643 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:03.299579 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad37ca3-afcc-4cb7-b3e4-4703356b0643 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:03.303529 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad37ca3-afcc-4cb7-b3e4-4703356b0643 service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:03.325425 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 923/1843] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:46:03 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:46:03.327095 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731cb2f2-88ad-4860-b118-a55a2a9a7c88 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:03.327494 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731cb2f2-88ad-4860-b118-a55a2a9a7c88 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:03.328867 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731cb2f2-88ad-4860-b118-a55a2a9a7c88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:03.381636 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8847bab-e007-47df-8704-681fd5d75cd7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:03.384401 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8847bab-e007-47df-8704-681fd5d75cd7 service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:03.384691 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8847bab-e007-47df-8704-681fd5d75cd7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:03.384976 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8847bab-e007-47df-8704-681fd5d75cd7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:03.403293 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8847bab-e007-47df-8704-681fd5d75cd7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:03.405836 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-731cb2f2-88ad-4860-b118-a55a2a9a7c88 None None] MFA Rules not processed for user `62e39f0e73c14dc0928445355f277de3`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:46:03.406448 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8847bab-e007-47df-8704-681fd5d75cd7 service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:03.409971 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-731cb2f2-88ad-4860-b118-a55a2a9a7c88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:46:03.425951 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 921/1844] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:46:03 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:46:03.431027 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8847bab-e007-47df-8704-681fd5d75cd7 service neutron] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:03.435974 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-d8847bab-e007-47df-8704-681fd5d75cd7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:46:03.523679 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 924/1845] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:46:03 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:46:03.529442 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731cb2f2-88ad-4860-b118-a55a2a9a7c88 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:03.557150 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8847bab-e007-47df-8704-681fd5d75cd7 service neutron] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:03.573589 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd68814-9cbf-4625-99de-aab0f093df01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:03.574927 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd68814-9cbf-4625-99de-aab0f093df01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:03.575408 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd68814-9cbf-4625-99de-aab0f093df01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:03.575846 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd68814-9cbf-4625-99de-aab0f093df01 admin admin] PATH_INFO: `/v3/projects/0d5d606a271547fc88fd98b31e3bf492` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:03.577412 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd68814-9cbf-4625-99de-aab0f093df01 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d5d606a271547fc88fd98b31e3bf492)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:03.580055 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd68814-9cbf-4625-99de-aab0f093df01 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:03.633332 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731cb2f2-88ad-4860-b118-a55a2a9a7c88 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:03.641100 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e549dea-9bc9-4dd9-8a5f-e5a7f7e2b08e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:03.646034 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e549dea-9bc9-4dd9-8a5f-e5a7f7e2b08e admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:03.646034 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e549dea-9bc9-4dd9-8a5f-e5a7f7e2b08e admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:03.646034 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e549dea-9bc9-4dd9-8a5f-e5a7f7e2b08e admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:03.646034 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e549dea-9bc9-4dd9-8a5f-e5a7f7e2b08e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:03.646034 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e549dea-9bc9-4dd9-8a5f-e5a7f7e2b08e admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:03.772058 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-4cd68814-9cbf-4625-99de-aab0f093df01 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d5d606a271547fc88fd98b31e3bf492', 'request_id': 'req-4cd68814-9cbf-4625-99de-aab0f093df01'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:03.775554 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 925/1846] 10.222.0.81 () {60 vars in 1152 bytes} [Wed Jul 20 22:46:03 2022] POST /identity/v3/projects => generated 383 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 22:46:03.779716 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e549dea-9bc9-4dd9-8a5f-e5a7f7e2b08e admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:03.849044 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-4cd68814-9cbf-4625-99de-aab0f093df01 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d5d606a271547fc88fd98b31e3bf492', 'request_id': 'req-4cd68814-9cbf-4625-99de-aab0f093df01'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:03.857155 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-4cd68814-9cbf-4625-99de-aab0f093df01 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d5d606a271547fc88fd98b31e3bf492', 'request_id': 'req-4cd68814-9cbf-4625-99de-aab0f093df01'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:03.898694 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-4cd68814-9cbf-4625-99de-aab0f093df01 admin admin] The token cache is being invalidate because project 0d5d606a271547fc88fd98b31e3bf492 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:46:03.899490 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-4cd68814-9cbf-4625-99de-aab0f093df01 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:03.902223 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-4cd68814-9cbf-4625-99de-aab0f093df01 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:03.904990 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 922/1847] 10.222.0.81 () {58 vars in 1232 bytes} [Wed Jul 20 22:46:03 2022] DELETE /identity/v3/projects/0d5d606a271547fc88fd98b31e3bf492 => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:46:03.912563 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e55381e-2809-4da1-8eee-fbb7390f02a4 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:03.912904 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e55381e-2809-4da1-8eee-fbb7390f02a4 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:03.913210 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e55381e-2809-4da1-8eee-fbb7390f02a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:03.936849 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-1e55381e-2809-4da1-8eee-fbb7390f02a4 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:46:03.938090 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-1e55381e-2809-4da1-8eee-fbb7390f02a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:46:03.957669 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e549dea-9bc9-4dd9-8a5f-e5a7f7e2b08e admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:03.964274 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004d9233-2b93-4541-97e7-4038aa6fe2e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:03.965399 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004d9233-2b93-4541-97e7-4038aa6fe2e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:03.965910 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004d9233-2b93-4541-97e7-4038aa6fe2e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:03.966388 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004d9233-2b93-4541-97e7-4038aa6fe2e7 admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:03.967347 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004d9233-2b93-4541-97e7-4038aa6fe2e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:03.970438 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004d9233-2b93-4541-97e7-4038aa6fe2e7 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:04.046385 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 923/1848] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:46:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:46:04.049480 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f016ec-08ca-4191-b3f6-cb68867d700e None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:04.049779 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f016ec-08ca-4191-b3f6-cb68867d700e None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:04.050060 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f016ec-08ca-4191-b3f6-cb68867d700e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:04.051889 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 926/1849] 10.222.0.81 () {60 vars in 1143 bytes} [Wed Jul 20 22:46:03 2022] POST /identity/v3/users => generated 298 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 22:46:04.056144 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004d9233-2b93-4541-97e7-4038aa6fe2e7 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:04.061179 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-004d9233-2b93-4541-97e7-4038aa6fe2e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:46:04.084136 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004d9233-2b93-4541-97e7-4038aa6fe2e7 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:04.098852 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-94f016ec-08ca-4191-b3f6-cb68867d700e None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:46:04.099859 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c463400-ffd0-4a59-91df-37b191f4e121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:04.101510 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c463400-ffd0-4a59-91df-37b191f4e121 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:04.102041 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c463400-ffd0-4a59-91df-37b191f4e121 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:04.102580 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c463400-ffd0-4a59-91df-37b191f4e121 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:04.104057 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c463400-ffd0-4a59-91df-37b191f4e121 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:04.104657 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-94f016ec-08ca-4191-b3f6-cb68867d700e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:46:04.108298 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c463400-ffd0-4a59-91df-37b191f4e121 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:04.121340 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 924/1850] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:46:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:46:04.122383 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f016ec-08ca-4191-b3f6-cb68867d700e None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:04.134637 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f016ec-08ca-4191-b3f6-cb68867d700e None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:04.140583 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d960cb-83a0-4b95-885e-683298fe12fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:04.141463 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d960cb-83a0-4b95-885e-683298fe12fb admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:04.141769 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d960cb-83a0-4b95-885e-683298fe12fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:04.142104 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d960cb-83a0-4b95-885e-683298fe12fb admin admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:04.143651 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d960cb-83a0-4b95-885e-683298fe12fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:46:04.144076 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d960cb-83a0-4b95-885e-683298fe12fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:04.146951 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 927/1851] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:46:04 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 22:46:04.149017 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d960cb-83a0-4b95-885e-683298fe12fb admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:04.152733 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c463400-ffd0-4a59-91df-37b191f4e121 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:04.190929 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c463400-ffd0-4a59-91df-37b191f4e121 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:04.191928 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 925/1852] 10.222.0.81 () {58 vars in 1152 bytes} [Wed Jul 20 22:46:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:46:04.208612 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d960cb-83a0-4b95-885e-683298fe12fb admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:04.219939 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09148f4b-5360-40f7-894f-c599eebab66b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:04.222036 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09148f4b-5360-40f7-894f-c599eebab66b admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:04.222423 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09148f4b-5360-40f7-894f-c599eebab66b admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:04.222763 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09148f4b-5360-40f7-894f-c599eebab66b admin admin] PATH_INFO: `/v3/projects/422aeb48221340c28ae70c72719ac14d/users/5fc548e86ced47119c09b531139937e8/roles/d89b6c92d35a401a8517f1d47f6be843` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:04.239881 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d960cb-83a0-4b95-885e-683298fe12fb admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:04.251870 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1005c274-686f-4ab4-bb54-06a18b6864ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:04.254235 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1005c274-686f-4ab4-bb54-06a18b6864ce admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:04.254779 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1005c274-686f-4ab4-bb54-06a18b6864ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:04.255581 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1005c274-686f-4ab4-bb54-06a18b6864ce admin admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:04.256461 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1005c274-686f-4ab4-bb54-06a18b6864ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:04.262980 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1005c274-686f-4ab4-bb54-06a18b6864ce admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:04.270658 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09148f4b-5360-40f7-894f-c599eebab66b admin admin] RBAC: Authorizing `identity:create_grant(project_id=422aeb48221340c28ae70c72719ac14d, user_id=5fc548e86ced47119c09b531139937e8, role_id=d89b6c92d35a401a8517f1d47f6be843)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:04.274641 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09148f4b-5360-40f7-894f-c599eebab66b admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:04.377297 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 928/1853] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:46:04 2022] PUT /identity/v3/projects/422aeb48221340c28ae70c72719ac14d/users/5fc548e86ced47119c09b531139937e8/roles/d89b6c92d35a401a8517f1d47f6be843 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:46:04.383893 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09148f4b-5360-40f7-894f-c599eebab66b admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:04.414574 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 926/1854] 10.222.0.81 () {60 vars in 1152 bytes} [Wed Jul 20 22:46:04 2022] POST /identity/v3/projects => generated 393 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:46:04.418727 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1005c274-686f-4ab4-bb54-06a18b6864ce admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:04.529615 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1005c274-686f-4ab4-bb54-06a18b6864ce admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:04.538970 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09148f4b-5360-40f7-894f-c599eebab66b admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:04.542223 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ceee784-aad3-4b9e-b3a1-5652b65fd832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:04.542960 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceee784-aad3-4b9e-b3a1-5652b65fd832 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:04.543293 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceee784-aad3-4b9e-b3a1-5652b65fd832 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:04.543577 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceee784-aad3-4b9e-b3a1-5652b65fd832 admin admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:04.544306 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ceee784-aad3-4b9e-b3a1-5652b65fd832 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:04.546436 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ceee784-aad3-4b9e-b3a1-5652b65fd832 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:04.554605 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fde67b5-d3af-4d9d-a402-2529b733c289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:04.555976 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fde67b5-d3af-4d9d-a402-2529b733c289 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:04.557049 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fde67b5-d3af-4d9d-a402-2529b733c289 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:04.557832 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fde67b5-d3af-4d9d-a402-2529b733c289 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:04.560121 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fde67b5-d3af-4d9d-a402-2529b733c289 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:04.567263 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fde67b5-d3af-4d9d-a402-2529b733c289 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:04.605869 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 929/1855] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:46:04 2022] GET /identity/v3/roles => generated 1420 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:46:04.611368 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fde67b5-d3af-4d9d-a402-2529b733c289 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:04.617709 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fde67b5-d3af-4d9d-a402-2529b733c289 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:04.623150 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be13d7e4-647c-4d5b-948a-772afa462cb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:04.625246 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be13d7e4-647c-4d5b-948a-772afa462cb2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:04.625246 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be13d7e4-647c-4d5b-948a-772afa462cb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:04.625246 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be13d7e4-647c-4d5b-948a-772afa462cb2 admin admin] PATH_INFO: `/v3/projects/422aeb48221340c28ae70c72719ac14d/users/5fc548e86ced47119c09b531139937e8/roles/d89b6c92d35a401a8517f1d47f6be843` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:04.627507 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be13d7e4-647c-4d5b-948a-772afa462cb2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=422aeb48221340c28ae70c72719ac14d, user_id=5fc548e86ced47119c09b531139937e8, role_id=d89b6c92d35a401a8517f1d47f6be843)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:04.632452 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be13d7e4-647c-4d5b-948a-772afa462cb2 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:04.648274 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 930/1856] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:46:04 2022] PUT /identity/v3/projects/422aeb48221340c28ae70c72719ac14d/users/5fc548e86ced47119c09b531139937e8/roles/d89b6c92d35a401a8517f1d47f6be843 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:46:04.660574 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9039510d-c304-4ddf-aaf8-46af00548260 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:04.661467 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9039510d-c304-4ddf-aaf8-46af00548260 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:04.661839 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9039510d-c304-4ddf-aaf8-46af00548260 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:04.691055 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 927/1857] 10.222.0.81 () {60 vars in 1143 bytes} [Wed Jul 20 22:46:04 2022] POST /identity/v3/users => generated 297 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 22:46:04.696566 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ceee784-aad3-4b9e-b3a1-5652b65fd832 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:04.718906 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ceee784-aad3-4b9e-b3a1-5652b65fd832 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:04.734057 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-9039510d-c304-4ddf-aaf8-46af00548260 None None] MFA Rules not processed for user `5fc548e86ced47119c09b531139937e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:46:04.735619 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-9039510d-c304-4ddf-aaf8-46af00548260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:46:04.790547 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710fcf28-a179-4d6d-aa00-aeb3a11f6fba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:04.792063 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710fcf28-a179-4d6d-aa00-aeb3a11f6fba admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:04.792534 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710fcf28-a179-4d6d-aa00-aeb3a11f6fba admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:04.792942 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710fcf28-a179-4d6d-aa00-aeb3a11f6fba admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:04.794331 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710fcf28-a179-4d6d-aa00-aeb3a11f6fba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:04.795713 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710fcf28-a179-4d6d-aa00-aeb3a11f6fba admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:04.811469 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 928/1858] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:46:04 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:46:04.813099 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710fcf28-a179-4d6d-aa00-aeb3a11f6fba admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:04.825364 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710fcf28-a179-4d6d-aa00-aeb3a11f6fba admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:04.831382 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de06db33-ef6d-42e3-b0e2-75ff15439c3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:04.832145 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de06db33-ef6d-42e3-b0e2-75ff15439c3d admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:04.832498 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de06db33-ef6d-42e3-b0e2-75ff15439c3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:04.832851 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de06db33-ef6d-42e3-b0e2-75ff15439c3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:04.833733 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de06db33-ef6d-42e3-b0e2-75ff15439c3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:04.835570 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de06db33-ef6d-42e3-b0e2-75ff15439c3d admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:04.852070 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 931/1859] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:46:04 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 22:46:04.864037 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 929/1860] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:46:04 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:46:04.867914 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9039510d-c304-4ddf-aaf8-46af00548260 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:04.876656 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de06db33-ef6d-42e3-b0e2-75ff15439c3d admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:04.879758 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-de06db33-ef6d-42e3-b0e2-75ff15439c3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:46:04.909496 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9039510d-c304-4ddf-aaf8-46af00548260 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:04.925620 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448c1a89-b7ee-4f76-a82f-23b9b180229d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:04.928255 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448c1a89-b7ee-4f76-a82f-23b9b180229d admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:04.928624 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448c1a89-b7ee-4f76-a82f-23b9b180229d admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:04.929823 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448c1a89-b7ee-4f76-a82f-23b9b180229d admin admin] PATH_INFO: `/v3/projects/889788e3066c4750af92fedde8274a1d/users/b996e4ffad114f1da93a4a546c200bcd/roles/559942c4de1744169f8b5e0404ecde6b` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:04.953949 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448c1a89-b7ee-4f76-a82f-23b9b180229d admin admin] RBAC: Authorizing `identity:create_grant(project_id=889788e3066c4750af92fedde8274a1d, user_id=b996e4ffad114f1da93a4a546c200bcd, role_id=559942c4de1744169f8b5e0404ecde6b)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:04.957696 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448c1a89-b7ee-4f76-a82f-23b9b180229d admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:05.013594 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de06db33-ef6d-42e3-b0e2-75ff15439c3d admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:05.022008 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1222862-4d5f-4fab-8fb2-635462aaf5d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecf4db289fba4e4a806198c62d31f7c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:05.022569 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1222862-4d5f-4fab-8fb2-635462aaf5d7 service cinder] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:05.022857 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1222862-4d5f-4fab-8fb2-635462aaf5d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:05.023169 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1222862-4d5f-4fab-8fb2-635462aaf5d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:05.046425 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1222862-4d5f-4fab-8fb2-635462aaf5d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:05.047935 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1222862-4d5f-4fab-8fb2-635462aaf5d7 service cinder] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:05.069126 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 930/1861] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:46:04 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:46:05.070503 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f85a3f-42d8-474e-a8be-10a09142b095 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:05.070793 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f85a3f-42d8-474e-a8be-10a09142b095 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:05.071113 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f85a3f-42d8-474e-a8be-10a09142b095 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:05.074214 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 931/1862] 10.222.0.81 () {54 vars in 921 bytes} [Wed Jul 20 22:46:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:46:05.078272 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b85999-0caa-490b-b6c8-bc4efe3f53e7 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:05.078565 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b85999-0caa-490b-b6c8-bc4efe3f53e7 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:05.078846 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b85999-0caa-490b-b6c8-bc4efe3f53e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:05.102403 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 932/1863] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:46:04 2022] PUT /identity/v3/projects/889788e3066c4750af92fedde8274a1d/users/b996e4ffad114f1da93a4a546c200bcd/roles/559942c4de1744169f8b5e0404ecde6b => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 22:46:05.114998 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f08eff-bae6-4986-bba0-f36ba594b84a None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:05.115344 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f08eff-bae6-4986-bba0-f36ba594b84a None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:05.115633 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f08eff-bae6-4986-bba0-f36ba594b84a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:05.126239 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-79b85999-0caa-490b-b6c8-bc4efe3f53e7 None None] MFA Rules not processed for user `7829324db2574d38b18c749d0559177f`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:46:05.127509 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-79b85999-0caa-490b-b6c8-bc4efe3f53e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:46:05.228516 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-e4f08eff-bae6-4986-bba0-f36ba594b84a None None] MFA Rules not processed for user `b996e4ffad114f1da93a4a546c200bcd`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:46:05.230052 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-e4f08eff-bae6-4986-bba0-f36ba594b84a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:46:05.393013 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 932/1864] 10.222.0.81 () {58 vars in 1018 bytes} [Wed Jul 20 22:46:05 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:46:05.394104 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b85999-0caa-490b-b6c8-bc4efe3f53e7 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:05.447629 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 933/1865] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:46:05 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 334 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:46:05.451695 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f08eff-bae6-4986-bba0-f36ba594b84a None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:05.561453 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b85999-0caa-490b-b6c8-bc4efe3f53e7 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:05.571883 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde8c1e0-c108-4fb4-8aea-ebfd6ca73eab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecf4db289fba4e4a806198c62d31f7c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:05.572654 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde8c1e0-c108-4fb4-8aea-ebfd6ca73eab service cinder] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:05.573298 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde8c1e0-c108-4fb4-8aea-ebfd6ca73eab service cinder] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:05.573609 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde8c1e0-c108-4fb4-8aea-ebfd6ca73eab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:05.576188 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-cde8c1e0-c108-4fb4-8aea-ebfd6ca73eab service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:46:05.609237 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=98,keystone:DELETE=1,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:46:05.646448 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f08eff-bae6-4986-bba0-f36ba594b84a None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:05.659990 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601197d9-fbe8-4188-8d6f-79556eb89b6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:05.660933 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601197d9-fbe8-4188-8d6f-79556eb89b6a admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:05.661368 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601197d9-fbe8-4188-8d6f-79556eb89b6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:05.661854 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601197d9-fbe8-4188-8d6f-79556eb89b6a admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:05.663865 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601197d9-fbe8-4188-8d6f-79556eb89b6a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:05.666618 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601197d9-fbe8-4188-8d6f-79556eb89b6a admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:05.753108 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:46:05.763783 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde8c1e0-c108-4fb4-8aea-ebfd6ca73eab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:05.766796 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde8c1e0-c108-4fb4-8aea-ebfd6ca73eab service cinder] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:05.794552 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 933/1866] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:46:05 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:46:05.807567 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 934/1867] 10.222.0.81 () {60 vars in 1152 bytes} [Wed Jul 20 22:46:05 2022] POST /identity/v3/projects => generated 391 bytes in 356 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:46:05.811292 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde8c1e0-c108-4fb4-8aea-ebfd6ca73eab service cinder] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:05.829130 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde8c1e0-c108-4fb4-8aea-ebfd6ca73eab service cinder] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:06.034815 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601197d9-fbe8-4188-8d6f-79556eb89b6a admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:06.037745 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-601197d9-fbe8-4188-8d6f-79556eb89b6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:46:06.048378 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6d4b0a-5ecc-4239-8300-a8cdf51625c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:06.049122 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6d4b0a-5ecc-4239-8300-a8cdf51625c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:06.049409 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6d4b0a-5ecc-4239-8300-a8cdf51625c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:06.049693 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6d4b0a-5ecc-4239-8300-a8cdf51625c1 admin admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:06.050401 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6d4b0a-5ecc-4239-8300-a8cdf51625c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:06.052437 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6d4b0a-5ecc-4239-8300-a8cdf51625c1 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:06.259327 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601197d9-fbe8-4188-8d6f-79556eb89b6a admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:06.270486 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136c6774-40db-4ce6-acea-33efe05f81bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7829324db2574d38b18c749d0559177f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:06.271207 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136c6774-40db-4ce6-acea-33efe05f81bc service nova] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:06.271501 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136c6774-40db-4ce6-acea-33efe05f81bc service nova] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:06.271787 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136c6774-40db-4ce6-acea-33efe05f81bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:06.285395 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136c6774-40db-4ce6-acea-33efe05f81bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:06.295664 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136c6774-40db-4ce6-acea-33efe05f81bc service nova] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:06.314911 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 935/1868] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:46:06 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 22:46:06.868787 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 934/1869] 10.222.0.81 () {60 vars in 1143 bytes} [Wed Jul 20 22:46:05 2022] POST /identity/v3/users => generated 302 bytes in 1057 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:46:06.874367 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136c6774-40db-4ce6-acea-33efe05f81bc service nova] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:06.885575 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136c6774-40db-4ce6-acea-33efe05f81bc service nova] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:06.892704 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0440eb-a043-4e4a-9ca2-19e858c11ab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:06.893669 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0440eb-a043-4e4a-9ca2-19e858c11ab3 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:06.894018 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0440eb-a043-4e4a-9ca2-19e858c11ab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:06.900171 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0440eb-a043-4e4a-9ca2-19e858c11ab3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:06.900171 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0440eb-a043-4e4a-9ca2-19e858c11ab3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:06.900171 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0440eb-a043-4e4a-9ca2-19e858c11ab3 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:06.915155 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 936/1870] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:46:06 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:46:06.918142 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6d4b0a-5ecc-4239-8300-a8cdf51625c1 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:06.931064 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6d4b0a-5ecc-4239-8300-a8cdf51625c1 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:06.936272 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b7813d-6b92-403c-aac1-7a65d50918d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:06.937588 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b7813d-6b92-403c-aac1-7a65d50918d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:06.938456 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b7813d-6b92-403c-aac1-7a65d50918d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:06.938911 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b7813d-6b92-403c-aac1-7a65d50918d7 admin admin] PATH_INFO: `/v3/projects/e420c7d338214be4aa14f256024a6ef8/users/43e74a437c0e4cd8a9520849231ec4fb/roles/d89b6c92d35a401a8517f1d47f6be843` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:06.992964 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b7813d-6b92-403c-aac1-7a65d50918d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e420c7d338214be4aa14f256024a6ef8, user_id=43e74a437c0e4cd8a9520849231ec4fb, role_id=d89b6c92d35a401a8517f1d47f6be843)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:07.000556 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b7813d-6b92-403c-aac1-7a65d50918d7 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:07.013459 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4f7f39-7e1c-49aa-876e-597a570bfa1f None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:07.013841 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4f7f39-7e1c-49aa-876e-597a570bfa1f None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:07.014013 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4f7f39-7e1c-49aa-876e-597a570bfa1f None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:07.015460 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 937/1871] 10.222.0.81 () {54 vars in 921 bytes} [Wed Jul 20 22:46:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 22:46:07.039368 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb9017f-cac2-45a2-be87-c8d8863d7346 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:07.039942 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb9017f-cac2-45a2-be87-c8d8863d7346 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:07.040179 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb9017f-cac2-45a2-be87-c8d8863d7346 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:07.101882 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-abb9017f-cac2-45a2-be87-c8d8863d7346 None None] MFA Rules not processed for user `7829324db2574d38b18c749d0559177f`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:46:07.105084 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-abb9017f-cac2-45a2-be87-c8d8863d7346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:46:07.129673 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 935/1872] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:46:06 2022] PUT /identity/v3/projects/e420c7d338214be4aa14f256024a6ef8/users/43e74a437c0e4cd8a9520849231ec4fb/roles/d89b6c92d35a401a8517f1d47f6be843 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:46:07.133746 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b7813d-6b92-403c-aac1-7a65d50918d7 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:07.151393 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 938/1873] 10.222.0.81 () {58 vars in 1018 bytes} [Wed Jul 20 22:46:07 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:46:07.204831 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb9017f-cac2-45a2-be87-c8d8863d7346 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:07.206605 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-abb9017f-cac2-45a2-be87-c8d8863d7346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:46:07.228273 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb9017f-cac2-45a2-be87-c8d8863d7346 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:07.278149 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b7813d-6b92-403c-aac1-7a65d50918d7 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:07.287793 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735a1f92-8521-4010-be5c-022adb1a7663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:07.290069 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735a1f92-8521-4010-be5c-022adb1a7663 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:07.290432 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735a1f92-8521-4010-be5c-022adb1a7663 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:07.290854 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735a1f92-8521-4010-be5c-022adb1a7663 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:07.291715 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735a1f92-8521-4010-be5c-022adb1a7663 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:07.293733 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735a1f92-8521-4010-be5c-022adb1a7663 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:07.337934 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 936/1874] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:46:07 2022] GET /identity/v3/roles => generated 1420 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 22:46:07.344343 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735a1f92-8521-4010-be5c-022adb1a7663 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:07.354146 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735a1f92-8521-4010-be5c-022adb1a7663 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:07.366364 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862f93b5-0e45-4f3c-a0dc-ea7c32789c4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:07.367341 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862f93b5-0e45-4f3c-a0dc-ea7c32789c4a admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:07.367631 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862f93b5-0e45-4f3c-a0dc-ea7c32789c4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:07.371947 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862f93b5-0e45-4f3c-a0dc-ea7c32789c4a admin admin] PATH_INFO: `/v3/projects/e420c7d338214be4aa14f256024a6ef8/users/43e74a437c0e4cd8a9520849231ec4fb/roles/d89b6c92d35a401a8517f1d47f6be843` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:07.382700 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862f93b5-0e45-4f3c-a0dc-ea7c32789c4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e420c7d338214be4aa14f256024a6ef8, user_id=43e74a437c0e4cd8a9520849231ec4fb, role_id=d89b6c92d35a401a8517f1d47f6be843)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:07.385467 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862f93b5-0e45-4f3c-a0dc-ea7c32789c4a admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:07.400441 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeaafe6-e179-48d0-b7cd-d63a25ebf23f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7829324db2574d38b18c749d0559177f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:07.412911 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeaafe6-e179-48d0-b7cd-d63a25ebf23f service nova] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:07.412911 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeaafe6-e179-48d0-b7cd-d63a25ebf23f service nova] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:07.412911 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeaafe6-e179-48d0-b7cd-d63a25ebf23f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:07.417054 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 937/1875] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:46:07 2022] PUT /identity/v3/projects/e420c7d338214be4aa14f256024a6ef8/users/43e74a437c0e4cd8a9520849231ec4fb/roles/d89b6c92d35a401a8517f1d47f6be843 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:46:07.430355 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceeaafe6-e179-48d0-b7cd-d63a25ebf23f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:07.430355 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceeaafe6-e179-48d0-b7cd-d63a25ebf23f service nova] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:07.437747 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fdcc11-e64f-4744-87d7-04600693e5dd None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:07.438043 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fdcc11-e64f-4744-87d7-04600693e5dd None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:07.438323 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fdcc11-e64f-4744-87d7-04600693e5dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:07.459348 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 939/1876] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:46:07 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:46:07.503823 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-e0fdcc11-e64f-4744-87d7-04600693e5dd None None] MFA Rules not processed for user `43e74a437c0e4cd8a9520849231ec4fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:46:07.505430 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-e0fdcc11-e64f-4744-87d7-04600693e5dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:46:07.686130 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 938/1877] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:46:07 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 22:46:07.688289 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeaafe6-e179-48d0-b7cd-d63a25ebf23f service nova] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:07.775218 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeaafe6-e179-48d0-b7cd-d63a25ebf23f service nova] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:07.852204 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d0fabd-a6df-48a9-8bf5-da1fb257660c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecf4db289fba4e4a806198c62d31f7c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:07.853161 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d0fabd-a6df-48a9-8bf5-da1fb257660c service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:07.854098 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d0fabd-a6df-48a9-8bf5-da1fb257660c service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:07.854384 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d0fabd-a6df-48a9-8bf5-da1fb257660c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:07.866193 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d0fabd-a6df-48a9-8bf5-da1fb257660c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:07.867745 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d0fabd-a6df-48a9-8bf5-da1fb257660c service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:07.889624 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 940/1878] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:46:07 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:46:07.952007 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fdcc11-e64f-4744-87d7-04600693e5dd None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:07.953615 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-e0fdcc11-e64f-4744-87d7-04600693e5dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:46:07.974641 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fdcc11-e64f-4744-87d7-04600693e5dd None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:07.981302 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf647b1-e4b6-4a03-9aaf-5daddf9e1252 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecf4db289fba4e4a806198c62d31f7c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:07.981990 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf647b1-e4b6-4a03-9aaf-5daddf9e1252 service cinder] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:07.982279 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf647b1-e4b6-4a03-9aaf-5daddf9e1252 service cinder] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:07.982634 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf647b1-e4b6-4a03-9aaf-5daddf9e1252 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:08.062794 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf647b1-e4b6-4a03-9aaf-5daddf9e1252 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:08.064384 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf647b1-e4b6-4a03-9aaf-5daddf9e1252 service cinder] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:08.074525 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 939/1879] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:46:07 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:46:08.877111 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:46:09.266299 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:46:10.185950 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:46:10.306236 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:46:10.756819 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:46:10.882664 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:46:11.174052 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:46:11.248514 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:46:11.482447 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:46:12.725945 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d0fabd-a6df-48a9-8bf5-da1fb257660c service cinder] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:12.734754 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d0fabd-a6df-48a9-8bf5-da1fb257660c service cinder] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:12.738417 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4175f36-4d15-4378-a561-2d7c8e56e724 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecf4db289fba4e4a806198c62d31f7c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:12.739110 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4175f36-4d15-4378-a561-2d7c8e56e724 service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:12.739400 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4175f36-4d15-4378-a561-2d7c8e56e724 service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:12.739694 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4175f36-4d15-4378-a561-2d7c8e56e724 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:12.741768 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-d4175f36-4d15-4378-a561-2d7c8e56e724 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:46:12.805547 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4175f36-4d15-4378-a561-2d7c8e56e724 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:12.806981 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4175f36-4d15-4378-a561-2d7c8e56e724 service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:12.816573 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 941/1880] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:46:12 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:46:14.501167 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf647b1-e4b6-4a03-9aaf-5daddf9e1252 service cinder] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:14.638208 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf647b1-e4b6-4a03-9aaf-5daddf9e1252 service cinder] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:14.662128 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eedbbe9-4a8e-4001-b6e6-1d417e63fded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:14.672315 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eedbbe9-4a8e-4001-b6e6-1d417e63fded admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:14.672315 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eedbbe9-4a8e-4001-b6e6-1d417e63fded admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:14.672315 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eedbbe9-4a8e-4001-b6e6-1d417e63fded admin admin] PATH_INFO: `/v3/users/62e39f0e73c14dc0928445355f277de3` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:14.672315 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eedbbe9-4a8e-4001-b6e6-1d417e63fded admin admin] RBAC: Authorizing `identity:delete_user(user_id=62e39f0e73c14dc0928445355f277de3)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:14.672315 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eedbbe9-4a8e-4001-b6e6-1d417e63fded admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:14.939752 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-4eedbbe9-4a8e-4001-b6e6-1d417e63fded admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62e39f0e73c14dc0928445355f277de3'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:14.940428 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-4eedbbe9-4a8e-4001-b6e6-1d417e63fded admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62e39f0e73c14dc0928445355f277de3'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:14.941042 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-4eedbbe9-4a8e-4001-b6e6-1d417e63fded admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62e39f0e73c14dc0928445355f277de3'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:14.948611 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-4eedbbe9-4a8e-4001-b6e6-1d417e63fded admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62e39f0e73c14dc0928445355f277de3'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:15.065096 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-4eedbbe9-4a8e-4001-b6e6-1d417e63fded admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62e39f0e73c14dc0928445355f277de3'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:15.087702 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 940/1881] 10.222.0.81 () {58 vars in 1223 bytes} [Wed Jul 20 22:46:14 2022] DELETE /identity/v3/users/62e39f0e73c14dc0928445355f277de3 => generated 0 bytes in 587 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:46:15.092332 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67146ee-8a41-4375-a50f-60adb471d8f4 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:15.093327 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67146ee-8a41-4375-a50f-60adb471d8f4 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:15.093623 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67146ee-8a41-4375-a50f-60adb471d8f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:15.127131 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-b67146ee-8a41-4375-a50f-60adb471d8f4 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:46:15.128577 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-b67146ee-8a41-4375-a50f-60adb471d8f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:46:15.238019 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 942/1882] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:46:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:46:15.245470 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eedbbe9-4a8e-4001-b6e6-1d417e63fded admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:15.247247 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-4eedbbe9-4a8e-4001-b6e6-1d417e63fded admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:46:15.290467 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:46:15.325538 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eedbbe9-4a8e-4001-b6e6-1d417e63fded admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:15.330980 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8941f3cd-e06b-4146-907f-ac18adfddc80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:15.331735 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8941f3cd-e06b-4146-907f-ac18adfddc80 service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:15.332022 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8941f3cd-e06b-4146-907f-ac18adfddc80 service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:15.332325 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8941f3cd-e06b-4146-907f-ac18adfddc80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:15.343541 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8941f3cd-e06b-4146-907f-ac18adfddc80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:15.345065 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8941f3cd-e06b-4146-907f-ac18adfddc80 service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:15.367292 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 941/1883] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:46:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:46:15.925376 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67146ee-8a41-4375-a50f-60adb471d8f4 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:15.926988 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-b67146ee-8a41-4375-a50f-60adb471d8f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:46:15.939766 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67146ee-8a41-4375-a50f-60adb471d8f4 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:15.945202 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4495306b-436c-4507-b739-de8f5f10bd7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:15.946138 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4495306b-436c-4507-b739-de8f5f10bd7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:15.946448 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4495306b-436c-4507-b739-de8f5f10bd7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:15.946688 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4495306b-436c-4507-b739-de8f5f10bd7f admin admin] PATH_INFO: `/v3/projects/edc43b27235c403098ea4232fc152c8c` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:15.947982 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4495306b-436c-4507-b739-de8f5f10bd7f admin admin] RBAC: Authorizing `identity:delete_project(project_id=edc43b27235c403098ea4232fc152c8c)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:15.950576 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4495306b-436c-4507-b739-de8f5f10bd7f admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:16.105872 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-4495306b-436c-4507-b739-de8f5f10bd7f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'edc43b27235c403098ea4232fc152c8c', 'request_id': 'req-4495306b-436c-4507-b739-de8f5f10bd7f'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:16.112938 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-4495306b-436c-4507-b739-de8f5f10bd7f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'edc43b27235c403098ea4232fc152c8c', 'request_id': 'req-4495306b-436c-4507-b739-de8f5f10bd7f'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:16.115507 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:46:16.173549 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-4495306b-436c-4507-b739-de8f5f10bd7f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'edc43b27235c403098ea4232fc152c8c', 'request_id': 'req-4495306b-436c-4507-b739-de8f5f10bd7f'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:16.184456 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-4495306b-436c-4507-b739-de8f5f10bd7f admin admin] The token cache is being invalidate because project edc43b27235c403098ea4232fc152c8c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:46:16.184834 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-4495306b-436c-4507-b739-de8f5f10bd7f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:16.185618 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-4495306b-436c-4507-b739-de8f5f10bd7f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:16.188006 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 943/1884] 10.222.0.81 () {58 vars in 1232 bytes} [Wed Jul 20 22:46:15 2022] DELETE /identity/v3/projects/edc43b27235c403098ea4232fc152c8c => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:46:16.192136 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8941f3cd-e06b-4146-907f-ac18adfddc80 service neutron] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:16.202198 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8941f3cd-e06b-4146-907f-ac18adfddc80 service neutron] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:16.257653 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df367d7e-0109-482a-9e86-ffc5ab52a4f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:16.258550 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df367d7e-0109-482a-9e86-ffc5ab52a4f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:16.258838 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df367d7e-0109-482a-9e86-ffc5ab52a4f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:16.259158 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df367d7e-0109-482a-9e86-ffc5ab52a4f2 admin admin] PATH_INFO: `/v3/users/5fc548e86ced47119c09b531139937e8` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:16.260348 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df367d7e-0109-482a-9e86-ffc5ab52a4f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5fc548e86ced47119c09b531139937e8)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:16.262314 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df367d7e-0109-482a-9e86-ffc5ab52a4f2 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:16.530072 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-df367d7e-0109-482a-9e86-ffc5ab52a4f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5fc548e86ced47119c09b531139937e8'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:16.536608 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-df367d7e-0109-482a-9e86-ffc5ab52a4f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5fc548e86ced47119c09b531139937e8'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:16.536608 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-df367d7e-0109-482a-9e86-ffc5ab52a4f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5fc548e86ced47119c09b531139937e8'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:16.539593 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-df367d7e-0109-482a-9e86-ffc5ab52a4f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5fc548e86ced47119c09b531139937e8'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:16.636727 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-df367d7e-0109-482a-9e86-ffc5ab52a4f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5fc548e86ced47119c09b531139937e8'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:16.656811 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 942/1885] 10.222.0.81 () {58 vars in 1223 bytes} [Wed Jul 20 22:46:16 2022] DELETE /identity/v3/users/5fc548e86ced47119c09b531139937e8 => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:46:17.118391 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4495306b-436c-4507-b739-de8f5f10bd7f admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:17.123218 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-4495306b-436c-4507-b739-de8f5f10bd7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:46:17.132940 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4495306b-436c-4507-b739-de8f5f10bd7f admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:17.197375 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6ca47f-a794-40e9-a9e1-eee880439e13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:17.197777 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6ca47f-a794-40e9-a9e1-eee880439e13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:17.198060 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6ca47f-a794-40e9-a9e1-eee880439e13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:17.198335 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6ca47f-a794-40e9-a9e1-eee880439e13 admin admin] PATH_INFO: `/v3/projects/422aeb48221340c28ae70c72719ac14d` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:17.199573 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6ca47f-a794-40e9-a9e1-eee880439e13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=422aeb48221340c28ae70c72719ac14d)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:17.201586 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6ca47f-a794-40e9-a9e1-eee880439e13 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:17.316540 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-7c6ca47f-a794-40e9-a9e1-eee880439e13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '422aeb48221340c28ae70c72719ac14d', 'request_id': 'req-7c6ca47f-a794-40e9-a9e1-eee880439e13'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:17.340721 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-7c6ca47f-a794-40e9-a9e1-eee880439e13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '422aeb48221340c28ae70c72719ac14d', 'request_id': 'req-7c6ca47f-a794-40e9-a9e1-eee880439e13'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:17.413458 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-7c6ca47f-a794-40e9-a9e1-eee880439e13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '422aeb48221340c28ae70c72719ac14d', 'request_id': 'req-7c6ca47f-a794-40e9-a9e1-eee880439e13'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:17.424866 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-7c6ca47f-a794-40e9-a9e1-eee880439e13 admin admin] The token cache is being invalidate because project 422aeb48221340c28ae70c72719ac14d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:46:17.425158 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-7c6ca47f-a794-40e9-a9e1-eee880439e13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:17.425801 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-7c6ca47f-a794-40e9-a9e1-eee880439e13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:17.427381 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 944/1886] 10.222.0.81 () {58 vars in 1232 bytes} [Wed Jul 20 22:46:17 2022] DELETE /identity/v3/projects/422aeb48221340c28ae70c72719ac14d => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:46:17.432229 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02fecbf-6ce5-44a4-b2c3-dd009eabc018 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:17.432687 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02fecbf-6ce5-44a4-b2c3-dd009eabc018 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:17.432805 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02fecbf-6ce5-44a4-b2c3-dd009eabc018 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:17.451299 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-c02fecbf-6ce5-44a4-b2c3-dd009eabc018 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:46:17.452674 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-c02fecbf-6ce5-44a4-b2c3-dd009eabc018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:46:17.493353 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:46:17.512579 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 943/1887] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:46:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:46:17.520591 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53789c0f-6d5c-4528-9ddd-12a31479c007 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:17.520881 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53789c0f-6d5c-4528-9ddd-12a31479c007 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:17.521160 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53789c0f-6d5c-4528-9ddd-12a31479c007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:17.539598 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-53789c0f-6d5c-4528-9ddd-12a31479c007 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:46:17.540864 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-53789c0f-6d5c-4528-9ddd-12a31479c007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:46:17.549630 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 945/1888] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:46:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:46:17.553976 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02fecbf-6ce5-44a4-b2c3-dd009eabc018 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:17.564355 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02fecbf-6ce5-44a4-b2c3-dd009eabc018 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:17.569034 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf752b77-ffb5-46ab-baec-9ed43ec2dffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:17.569693 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf752b77-ffb5-46ab-baec-9ed43ec2dffd admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:17.569974 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf752b77-ffb5-46ab-baec-9ed43ec2dffd admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:17.570249 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf752b77-ffb5-46ab-baec-9ed43ec2dffd admin admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:17.571002 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf752b77-ffb5-46ab-baec-9ed43ec2dffd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:46:17.571336 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf752b77-ffb5-46ab-baec-9ed43ec2dffd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:17.572976 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf752b77-ffb5-46ab-baec-9ed43ec2dffd admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:17.589071 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 944/1889] 10.222.0.81 () {58 vars in 1152 bytes} [Wed Jul 20 22:46:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:46:17.592489 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53789c0f-6d5c-4528-9ddd-12a31479c007 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:17.606805 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53789c0f-6d5c-4528-9ddd-12a31479c007 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:17.614964 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6efb26-cfda-41bc-ac08-aa19477a62ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:17.616037 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6efb26-cfda-41bc-ac08-aa19477a62ca admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:17.616507 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6efb26-cfda-41bc-ac08-aa19477a62ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:17.617022 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6efb26-cfda-41bc-ac08-aa19477a62ca admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:17.618191 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6efb26-cfda-41bc-ac08-aa19477a62ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:17.621564 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6efb26-cfda-41bc-ac08-aa19477a62ca admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:17.732095 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 946/1890] 10.222.0.81 () {60 vars in 1152 bytes} [Wed Jul 20 22:46:17 2022] POST /identity/v3/projects => generated 367 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:46:17.736223 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf752b77-ffb5-46ab-baec-9ed43ec2dffd admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:17.746458 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf752b77-ffb5-46ab-baec-9ed43ec2dffd admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:17.804166 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15460d2-d1e4-477a-8c4b-f3b393f8ac07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:17.804840 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15460d2-d1e4-477a-8c4b-f3b393f8ac07 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:17.805128 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15460d2-d1e4-477a-8c4b-f3b393f8ac07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:17.805403 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15460d2-d1e4-477a-8c4b-f3b393f8ac07 admin admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:17.806159 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15460d2-d1e4-477a-8c4b-f3b393f8ac07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:17.808149 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15460d2-d1e4-477a-8c4b-f3b393f8ac07 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:17.944324 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 945/1891] 10.222.0.81 () {60 vars in 1143 bytes} [Wed Jul 20 22:46:17 2022] POST /identity/v3/users => generated 284 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 22:46:17.947170 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6efb26-cfda-41bc-ac08-aa19477a62ca admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:17.958115 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6efb26-cfda-41bc-ac08-aa19477a62ca admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:17.963307 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e972d8-49de-4710-bd01-6b4a6449beda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:17.971158 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e972d8-49de-4710-bd01-6b4a6449beda admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:17.971158 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e972d8-49de-4710-bd01-6b4a6449beda admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:17.971158 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e972d8-49de-4710-bd01-6b4a6449beda admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:17.971158 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e972d8-49de-4710-bd01-6b4a6449beda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:17.971158 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e972d8-49de-4710-bd01-6b4a6449beda admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:17.979447 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 947/1892] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:46:17 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:46:17.985159 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15460d2-d1e4-477a-8c4b-f3b393f8ac07 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:17.997218 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15460d2-d1e4-477a-8c4b-f3b393f8ac07 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:18.002112 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c15e75f-6cd6-4054-867f-c1c873ba75f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:18.002821 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c15e75f-6cd6-4054-867f-c1c873ba75f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:18.003147 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c15e75f-6cd6-4054-867f-c1c873ba75f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:18.003434 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c15e75f-6cd6-4054-867f-c1c873ba75f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:18.004176 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c15e75f-6cd6-4054-867f-c1c873ba75f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:18.007147 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c15e75f-6cd6-4054-867f-c1c873ba75f7 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:18.019316 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 946/1893] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:46:17 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:46:18.024388 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e972d8-49de-4710-bd01-6b4a6449beda admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:18.036646 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e972d8-49de-4710-bd01-6b4a6449beda admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:18.044704 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d44fc7e-fea3-4b83-b581-710570155ebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:18.045779 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d44fc7e-fea3-4b83-b581-710570155ebc admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:18.046759 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d44fc7e-fea3-4b83-b581-710570155ebc admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:18.047685 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d44fc7e-fea3-4b83-b581-710570155ebc admin admin] PATH_INFO: `/v3/projects/a243aa5fdb324a67bcd1d7e8b78ed3d1/users/c05c4f58865c491b81dab0e5af07ce4f/roles/559942c4de1744169f8b5e0404ecde6b` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:18.063883 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d44fc7e-fea3-4b83-b581-710570155ebc admin admin] RBAC: Authorizing `identity:create_grant(project_id=a243aa5fdb324a67bcd1d7e8b78ed3d1, user_id=c05c4f58865c491b81dab0e5af07ce4f, role_id=559942c4de1744169f8b5e0404ecde6b)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:18.066784 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d44fc7e-fea3-4b83-b581-710570155ebc admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:18.247812 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 948/1894] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:46:18 2022] PUT /identity/v3/projects/a243aa5fdb324a67bcd1d7e8b78ed3d1/users/c05c4f58865c491b81dab0e5af07ce4f/roles/559942c4de1744169f8b5e0404ecde6b => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:46:18.254153 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6892269a-ea28-47ec-b0f7-83101f2885c8 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:18.254451 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6892269a-ea28-47ec-b0f7-83101f2885c8 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:18.255370 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6892269a-ea28-47ec-b0f7-83101f2885c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:18.281874 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-6892269a-ea28-47ec-b0f7-83101f2885c8 None None] MFA Rules not processed for user `c05c4f58865c491b81dab0e5af07ce4f`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:46:18.283107 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-6892269a-ea28-47ec-b0f7-83101f2885c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:46:18.354033 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 947/1895] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:46:18 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:46:18.359982 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d44fc7e-fea3-4b83-b581-710570155ebc admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:18.373854 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:DELETE=10,keystone:SELECT=87,keystone:INSERT=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:46:18.423760 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d44fc7e-fea3-4b83-b581-710570155ebc admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:18.428526 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc83e634-7b11-4e44-9791-4dd8ee907f2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:18.429324 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc83e634-7b11-4e44-9791-4dd8ee907f2b admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:18.429554 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc83e634-7b11-4e44-9791-4dd8ee907f2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:18.430171 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc83e634-7b11-4e44-9791-4dd8ee907f2b admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:18.431252 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc83e634-7b11-4e44-9791-4dd8ee907f2b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:18.433878 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc83e634-7b11-4e44-9791-4dd8ee907f2b admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:18.506876 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 949/1896] 10.222.0.81 () {60 vars in 1152 bytes} [Wed Jul 20 22:46:18 2022] POST /identity/v3/projects => generated 367 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:46:18.511072 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6892269a-ea28-47ec-b0f7-83101f2885c8 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:18.522944 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6892269a-ea28-47ec-b0f7-83101f2885c8 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:18.562649 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:46:18.582806 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dc46d5-40d7-4b34-b7e8-38cbb80f7f08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:18.583570 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dc46d5-40d7-4b34-b7e8-38cbb80f7f08 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:18.583861 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dc46d5-40d7-4b34-b7e8-38cbb80f7f08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:18.584146 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dc46d5-40d7-4b34-b7e8-38cbb80f7f08 admin admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:18.584843 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dc46d5-40d7-4b34-b7e8-38cbb80f7f08 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:18.586760 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dc46d5-40d7-4b34-b7e8-38cbb80f7f08 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:18.734282 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 948/1897] 10.222.0.81 () {60 vars in 1143 bytes} [Wed Jul 20 22:46:18 2022] POST /identity/v3/users => generated 290 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:46:18.738052 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc83e634-7b11-4e44-9791-4dd8ee907f2b admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:18.748140 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc83e634-7b11-4e44-9791-4dd8ee907f2b admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:18.754148 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4199c8-47ff-4d92-9d8a-ff64e8d1f6e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:18.754148 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4199c8-47ff-4d92-9d8a-ff64e8d1f6e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:18.754423 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4199c8-47ff-4d92-9d8a-ff64e8d1f6e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:18.754740 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4199c8-47ff-4d92-9d8a-ff64e8d1f6e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:18.755552 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4199c8-47ff-4d92-9d8a-ff64e8d1f6e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:18.757496 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4199c8-47ff-4d92-9d8a-ff64e8d1f6e6 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:18.772524 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 950/1898] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:46:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 22:46:18.774094 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dc46d5-40d7-4b34-b7e8-38cbb80f7f08 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:18.785305 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dc46d5-40d7-4b34-b7e8-38cbb80f7f08 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:18.789926 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af035eab-704d-40b7-81f4-2a4628fb0f7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:18.790826 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af035eab-704d-40b7-81f4-2a4628fb0f7b admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:18.791148 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af035eab-704d-40b7-81f4-2a4628fb0f7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:18.797427 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af035eab-704d-40b7-81f4-2a4628fb0f7b admin admin] PATH_INFO: `/v3/projects/0048e079b1bc420a88be6b02d09f599b/users/abdb0384039b4e449e5f7ae2b45cca47/roles/d89b6c92d35a401a8517f1d47f6be843` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:18.814353 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af035eab-704d-40b7-81f4-2a4628fb0f7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0048e079b1bc420a88be6b02d09f599b, user_id=abdb0384039b4e449e5f7ae2b45cca47, role_id=d89b6c92d35a401a8517f1d47f6be843)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:18.817176 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af035eab-704d-40b7-81f4-2a4628fb0f7b admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:18.895042 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 949/1899] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:46:18 2022] PUT /identity/v3/projects/0048e079b1bc420a88be6b02d09f599b/users/abdb0384039b4e449e5f7ae2b45cca47/roles/d89b6c92d35a401a8517f1d47f6be843 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:46:18.899094 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4199c8-47ff-4d92-9d8a-ff64e8d1f6e6 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:18.910262 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4199c8-47ff-4d92-9d8a-ff64e8d1f6e6 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:18.970143 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b4d7ae-4df4-48c6-8f83-413348700ec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:18.970808 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b4d7ae-4df4-48c6-8f83-413348700ec3 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:18.971134 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b4d7ae-4df4-48c6-8f83-413348700ec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:18.971420 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b4d7ae-4df4-48c6-8f83-413348700ec3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:18.972142 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b4d7ae-4df4-48c6-8f83-413348700ec3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:18.973846 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b4d7ae-4df4-48c6-8f83-413348700ec3 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:18.985105 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 951/1900] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:46:18 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:46:18.989368 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af035eab-704d-40b7-81f4-2a4628fb0f7b admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:18.999831 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af035eab-704d-40b7-81f4-2a4628fb0f7b admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:19.008442 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e721a4-ab79-4f8c-8d2d-36bed04b1e30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:19.008442 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e721a4-ab79-4f8c-8d2d-36bed04b1e30 admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:19.008442 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e721a4-ab79-4f8c-8d2d-36bed04b1e30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:19.008442 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e721a4-ab79-4f8c-8d2d-36bed04b1e30 admin admin] PATH_INFO: `/v3/projects/0048e079b1bc420a88be6b02d09f599b/users/abdb0384039b4e449e5f7ae2b45cca47/roles/d89b6c92d35a401a8517f1d47f6be843` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:19.008798 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e721a4-ab79-4f8c-8d2d-36bed04b1e30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0048e079b1bc420a88be6b02d09f599b, user_id=abdb0384039b4e449e5f7ae2b45cca47, role_id=d89b6c92d35a401a8517f1d47f6be843)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:19.010799 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e721a4-ab79-4f8c-8d2d-36bed04b1e30 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:19.021166 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 950/1901] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:46:18 2022] PUT /identity/v3/projects/0048e079b1bc420a88be6b02d09f599b/users/abdb0384039b4e449e5f7ae2b45cca47/roles/d89b6c92d35a401a8517f1d47f6be843 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:46:19.032007 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630a2ed6-a4f6-493b-9cde-1b74e0e04c10 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:19.032320 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630a2ed6-a4f6-493b-9cde-1b74e0e04c10 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:19.032624 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630a2ed6-a4f6-493b-9cde-1b74e0e04c10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:19.061627 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-630a2ed6-a4f6-493b-9cde-1b74e0e04c10 None None] MFA Rules not processed for user `abdb0384039b4e449e5f7ae2b45cca47`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:46:19.062858 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-630a2ed6-a4f6-493b-9cde-1b74e0e04c10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:46:19.137776 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 952/1902] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:46:19 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:46:19.170543 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e721a4-ab79-4f8c-8d2d-36bed04b1e30 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:19.172576 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-b7e721a4-ab79-4f8c-8d2d-36bed04b1e30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:46:19.288759 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e721a4-ab79-4f8c-8d2d-36bed04b1e30 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:19.292769 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f383dac8-ae48-4303-a7dd-8ffc55ad1a30 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecf4db289fba4e4a806198c62d31f7c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:19.293718 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f383dac8-ae48-4303-a7dd-8ffc55ad1a30 service cinder] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:19.297946 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f383dac8-ae48-4303-a7dd-8ffc55ad1a30 service cinder] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:19.297946 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f383dac8-ae48-4303-a7dd-8ffc55ad1a30 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:19.304871 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f383dac8-ae48-4303-a7dd-8ffc55ad1a30 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:19.306522 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f383dac8-ae48-4303-a7dd-8ffc55ad1a30 service cinder] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:19.317446 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 951/1903] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:46:19 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:46:19.599322 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630a2ed6-a4f6-493b-9cde-1b74e0e04c10 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:19.610421 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630a2ed6-a4f6-493b-9cde-1b74e0e04c10 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:19.614174 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb852c44-66c1-40b0-b10a-8d11120d673e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecf4db289fba4e4a806198c62d31f7c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:19.614863 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb852c44-66c1-40b0-b10a-8d11120d673e service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:19.615195 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb852c44-66c1-40b0-b10a-8d11120d673e service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:19.615476 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb852c44-66c1-40b0-b10a-8d11120d673e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:19.620240 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:46:19.675128 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb852c44-66c1-40b0-b10a-8d11120d673e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:19.676575 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb852c44-66c1-40b0-b10a-8d11120d673e service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:19.685842 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 953/1904] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:46:19 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:46:20.532597 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631c3202-f218-43a3-9d26-e6b8a8ed6977 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:20.533066 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631c3202-f218-43a3-9d26-e6b8a8ed6977 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:20.533300 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631c3202-f218-43a3-9d26-e6b8a8ed6977 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:20.535371 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 952/1905] 10.222.0.81 () {54 vars in 921 bytes} [Wed Jul 20 22:46:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:46:20.543980 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee27e73-587e-447d-8252-01276f298dad None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:20.544389 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee27e73-587e-447d-8252-01276f298dad None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:20.545622 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee27e73-587e-447d-8252-01276f298dad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:20.563142 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-dee27e73-587e-447d-8252-01276f298dad None None] MFA Rules not processed for user `7829324db2574d38b18c749d0559177f`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:46:20.564378 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-dee27e73-587e-447d-8252-01276f298dad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:46:20.638561 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 954/1906] 10.222.0.81 () {58 vars in 1018 bytes} [Wed Jul 20 22:46:20 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 22:46:20.651571 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631c3202-f218-43a3-9d26-e6b8a8ed6977 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:20.655873 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-631c3202-f218-43a3-9d26-e6b8a8ed6977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:46:20.674314 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631c3202-f218-43a3-9d26-e6b8a8ed6977 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:20.680473 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c5fa5d-ec05-4b86-95e6-8e1376b4cef9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7829324db2574d38b18c749d0559177f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:20.681124 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c5fa5d-ec05-4b86-95e6-8e1376b4cef9 service nova] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:20.681433 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c5fa5d-ec05-4b86-95e6-8e1376b4cef9 service nova] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:20.681691 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c5fa5d-ec05-4b86-95e6-8e1376b4cef9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:20.691592 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c5fa5d-ec05-4b86-95e6-8e1376b4cef9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:20.693156 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c5fa5d-ec05-4b86-95e6-8e1376b4cef9 service nova] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:20.705630 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 953/1907] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:46:20 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:46:21.059065 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee27e73-587e-447d-8252-01276f298dad None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:21.060586 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-dee27e73-587e-447d-8252-01276f298dad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:46:21.184081 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee27e73-587e-447d-8252-01276f298dad None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:21.208587 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08e7b52-df06-436f-ab6e-fbfb754a05e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:21.210716 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08e7b52-df06-436f-ab6e-fbfb754a05e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:21.211751 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08e7b52-df06-436f-ab6e-fbfb754a05e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:21.212719 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08e7b52-df06-436f-ab6e-fbfb754a05e6 admin admin] PATH_INFO: `/v3/users/b996e4ffad114f1da93a4a546c200bcd` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:21.216772 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08e7b52-df06-436f-ab6e-fbfb754a05e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b996e4ffad114f1da93a4a546c200bcd)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:21.223123 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08e7b52-df06-436f-ab6e-fbfb754a05e6 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:21.665274 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:46:21.674515 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-d08e7b52-df06-436f-ab6e-fbfb754a05e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b996e4ffad114f1da93a4a546c200bcd'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:21.675667 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-d08e7b52-df06-436f-ab6e-fbfb754a05e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b996e4ffad114f1da93a4a546c200bcd'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:21.676309 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-d08e7b52-df06-436f-ab6e-fbfb754a05e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b996e4ffad114f1da93a4a546c200bcd'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:21.685898 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-d08e7b52-df06-436f-ab6e-fbfb754a05e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b996e4ffad114f1da93a4a546c200bcd'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:21.820982 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-d08e7b52-df06-436f-ab6e-fbfb754a05e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b996e4ffad114f1da93a4a546c200bcd'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:21.858258 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 955/1908] 10.222.0.81 () {58 vars in 1223 bytes} [Wed Jul 20 22:46:21 2022] DELETE /identity/v3/users/b996e4ffad114f1da93a4a546c200bcd => generated 0 bytes in 800 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:46:21.862779 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a30e5e-804f-4f16-b4e6-7347c342a04f None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:21.863264 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a30e5e-804f-4f16-b4e6-7347c342a04f None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:21.865040 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a30e5e-804f-4f16-b4e6-7347c342a04f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:21.894819 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-95a30e5e-804f-4f16-b4e6-7347c342a04f None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:46:21.897527 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-95a30e5e-804f-4f16-b4e6-7347c342a04f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:46:21.986655 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 954/1909] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:46:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:46:22.001519 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08e7b52-df06-436f-ab6e-fbfb754a05e6 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:22.003605 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-d08e7b52-df06-436f-ab6e-fbfb754a05e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:46:22.082911 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08e7b52-df06-436f-ab6e-fbfb754a05e6 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:22.088226 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1ae6cf-8777-4366-b9e9-ab808e17bf48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:22.088611 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1ae6cf-8777-4366-b9e9-ab808e17bf48 service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:22.089093 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1ae6cf-8777-4366-b9e9-ab808e17bf48 service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:22.089586 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1ae6cf-8777-4366-b9e9-ab808e17bf48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:22.100608 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1ae6cf-8777-4366-b9e9-ab808e17bf48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:22.102900 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1ae6cf-8777-4366-b9e9-ab808e17bf48 service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:22.126467 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 956/1910] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:46:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:46:22.764337 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a30e5e-804f-4f16-b4e6-7347c342a04f None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:22.767590 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-95a30e5e-804f-4f16-b4e6-7347c342a04f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:46:22.781834 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a30e5e-804f-4f16-b4e6-7347c342a04f None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:22.787631 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c742c8a-d386-4adf-a68b-62e96a2160b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:22.788520 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c742c8a-d386-4adf-a68b-62e96a2160b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:22.788802 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c742c8a-d386-4adf-a68b-62e96a2160b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:22.789080 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c742c8a-d386-4adf-a68b-62e96a2160b2 admin admin] PATH_INFO: `/v3/projects/889788e3066c4750af92fedde8274a1d` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:22.790262 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c742c8a-d386-4adf-a68b-62e96a2160b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=889788e3066c4750af92fedde8274a1d)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:22.801701 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c742c8a-d386-4adf-a68b-62e96a2160b2 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:22.809856 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:46:22.933262 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-7c742c8a-d386-4adf-a68b-62e96a2160b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '889788e3066c4750af92fedde8274a1d', 'request_id': 'req-7c742c8a-d386-4adf-a68b-62e96a2160b2'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:23.041687 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-7c742c8a-d386-4adf-a68b-62e96a2160b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '889788e3066c4750af92fedde8274a1d', 'request_id': 'req-7c742c8a-d386-4adf-a68b-62e96a2160b2'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:23.048209 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-7c742c8a-d386-4adf-a68b-62e96a2160b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '889788e3066c4750af92fedde8274a1d', 'request_id': 'req-7c742c8a-d386-4adf-a68b-62e96a2160b2'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:23.059654 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-7c742c8a-d386-4adf-a68b-62e96a2160b2 admin admin] The token cache is being invalidate because project 889788e3066c4750af92fedde8274a1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:46:23.059941 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-7c742c8a-d386-4adf-a68b-62e96a2160b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:23.060573 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-7c742c8a-d386-4adf-a68b-62e96a2160b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:23.062928 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 955/1911] 10.222.0.81 () {58 vars in 1232 bytes} [Wed Jul 20 22:46:22 2022] DELETE /identity/v3/projects/889788e3066c4750af92fedde8274a1d => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:46:23.067000 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1ae6cf-8777-4366-b9e9-ab808e17bf48 service neutron] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:23.077654 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1ae6cf-8777-4366-b9e9-ab808e17bf48 service neutron] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:23.159550 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46bf276-6a46-461f-89f4-72205e2be71b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:23.160656 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46bf276-6a46-461f-89f4-72205e2be71b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:23.160942 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46bf276-6a46-461f-89f4-72205e2be71b admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:23.161218 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46bf276-6a46-461f-89f4-72205e2be71b admin admin] PATH_INFO: `/v3/users/43e74a437c0e4cd8a9520849231ec4fb` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:23.162525 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46bf276-6a46-461f-89f4-72205e2be71b admin admin] RBAC: Authorizing `identity:delete_user(user_id=43e74a437c0e4cd8a9520849231ec4fb)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:23.164560 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46bf276-6a46-461f-89f4-72205e2be71b admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:23.416499 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-b46bf276-6a46-461f-89f4-72205e2be71b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43e74a437c0e4cd8a9520849231ec4fb'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:23.417176 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-b46bf276-6a46-461f-89f4-72205e2be71b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43e74a437c0e4cd8a9520849231ec4fb'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:23.417775 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-b46bf276-6a46-461f-89f4-72205e2be71b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43e74a437c0e4cd8a9520849231ec4fb'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:23.425713 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-b46bf276-6a46-461f-89f4-72205e2be71b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43e74a437c0e4cd8a9520849231ec4fb'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:23.429551 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:46:23.538867 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-b46bf276-6a46-461f-89f4-72205e2be71b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43e74a437c0e4cd8a9520849231ec4fb'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:23.558072 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 957/1912] 10.222.0.81 () {58 vars in 1223 bytes} [Wed Jul 20 22:46:23 2022] DELETE /identity/v3/users/43e74a437c0e4cd8a9520849231ec4fb => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:46:24.020296 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c742c8a-d386-4adf-a68b-62e96a2160b2 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:24.022569 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-7c742c8a-d386-4adf-a68b-62e96a2160b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:46:24.038868 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c742c8a-d386-4adf-a68b-62e96a2160b2 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:24.096423 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ce1953-4207-4114-831a-27692457ff22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:24.097303 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ce1953-4207-4114-831a-27692457ff22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:24.097633 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ce1953-4207-4114-831a-27692457ff22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:24.097912 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ce1953-4207-4114-831a-27692457ff22 admin admin] PATH_INFO: `/v3/projects/e420c7d338214be4aa14f256024a6ef8` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:24.099130 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ce1953-4207-4114-831a-27692457ff22 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e420c7d338214be4aa14f256024a6ef8)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:24.101119 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ce1953-4207-4114-831a-27692457ff22 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:24.238320 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-47ce1953-4207-4114-831a-27692457ff22 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e420c7d338214be4aa14f256024a6ef8', 'request_id': 'req-47ce1953-4207-4114-831a-27692457ff22'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:24.351594 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-47ce1953-4207-4114-831a-27692457ff22 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e420c7d338214be4aa14f256024a6ef8', 'request_id': 'req-47ce1953-4207-4114-831a-27692457ff22'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:24.361312 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-47ce1953-4207-4114-831a-27692457ff22 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e420c7d338214be4aa14f256024a6ef8', 'request_id': 'req-47ce1953-4207-4114-831a-27692457ff22'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:24.377043 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-47ce1953-4207-4114-831a-27692457ff22 admin admin] The token cache is being invalidate because project e420c7d338214be4aa14f256024a6ef8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:46:24.377567 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-47ce1953-4207-4114-831a-27692457ff22 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:24.378566 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-47ce1953-4207-4114-831a-27692457ff22 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:24.380811 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 956/1913] 10.222.0.81 () {58 vars in 1232 bytes} [Wed Jul 20 22:46:24 2022] DELETE /identity/v3/projects/e420c7d338214be4aa14f256024a6ef8 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:46:24.385762 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf0d8f0-32a3-43d3-b118-cff90b52e5ff None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:24.386592 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf0d8f0-32a3-43d3-b118-cff90b52e5ff None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:24.387037 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf0d8f0-32a3-43d3-b118-cff90b52e5ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:24.414527 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-4bf0d8f0-32a3-43d3-b118-cff90b52e5ff None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:46:24.416012 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-4bf0d8f0-32a3-43d3-b118-cff90b52e5ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:46:24.497176 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 958/1914] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:46:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:46:24.505374 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589b750d-e5ca-44c1-93d7-f11a6db923b2 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:24.505667 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589b750d-e5ca-44c1-93d7-f11a6db923b2 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:24.506810 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589b750d-e5ca-44c1-93d7-f11a6db923b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:24.532541 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-589b750d-e5ca-44c1-93d7-f11a6db923b2 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:46:24.532541 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-589b750d-e5ca-44c1-93d7-f11a6db923b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:46:24.541929 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 957/1915] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:46:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:46:24.546164 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf0d8f0-32a3-43d3-b118-cff90b52e5ff None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:24.558012 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf0d8f0-32a3-43d3-b118-cff90b52e5ff None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:24.570132 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50523ec-cd1a-4c4c-938c-096987deeb9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:24.570132 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50523ec-cd1a-4c4c-938c-096987deeb9a admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:24.570132 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50523ec-cd1a-4c4c-938c-096987deeb9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:24.570132 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50523ec-cd1a-4c4c-938c-096987deeb9a admin admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:24.570458 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50523ec-cd1a-4c4c-938c-096987deeb9a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:46:24.570458 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50523ec-cd1a-4c4c-938c-096987deeb9a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:24.570458 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50523ec-cd1a-4c4c-938c-096987deeb9a admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:24.586257 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 959/1916] 10.222.0.81 () {58 vars in 1152 bytes} [Wed Jul 20 22:46:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:46:24.587875 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589b750d-e5ca-44c1-93d7-f11a6db923b2 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:24.598173 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589b750d-e5ca-44c1-93d7-f11a6db923b2 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:24.602930 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742be68a-1066-41bd-aff9-aa6d5a6b41b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:24.603608 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742be68a-1066-41bd-aff9-aa6d5a6b41b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:24.603896 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742be68a-1066-41bd-aff9-aa6d5a6b41b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:24.604180 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742be68a-1066-41bd-aff9-aa6d5a6b41b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:24.604901 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742be68a-1066-41bd-aff9-aa6d5a6b41b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:24.606819 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742be68a-1066-41bd-aff9-aa6d5a6b41b0 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:24.758172 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 958/1917] 10.222.0.81 () {60 vars in 1152 bytes} [Wed Jul 20 22:46:24 2022] POST /identity/v3/projects => generated 401 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:46:24.762390 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50523ec-cd1a-4c4c-938c-096987deeb9a admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:24.773147 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50523ec-cd1a-4c4c-938c-096987deeb9a admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:24.829297 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3b6406-de46-4483-9a7b-83b23f34b6c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:24.829982 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3b6406-de46-4483-9a7b-83b23f34b6c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:24.830270 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3b6406-de46-4483-9a7b-83b23f34b6c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:24.830553 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3b6406-de46-4483-9a7b-83b23f34b6c7 admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:24.831291 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3b6406-de46-4483-9a7b-83b23f34b6c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:24.833215 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3b6406-de46-4483-9a7b-83b23f34b6c7 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:24.962230 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 960/1918] 10.222.0.81 () {60 vars in 1143 bytes} [Wed Jul 20 22:46:24 2022] POST /identity/v3/users => generated 301 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:46:24.966539 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742be68a-1066-41bd-aff9-aa6d5a6b41b0 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:24.976744 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742be68a-1066-41bd-aff9-aa6d5a6b41b0 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:24.981488 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10adffb8-4f2d-43e0-8932-208a408c51f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:24.982230 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10adffb8-4f2d-43e0-8932-208a408c51f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:24.982590 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10adffb8-4f2d-43e0-8932-208a408c51f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:24.982947 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10adffb8-4f2d-43e0-8932-208a408c51f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:24.983715 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10adffb8-4f2d-43e0-8932-208a408c51f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:24.985324 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10adffb8-4f2d-43e0-8932-208a408c51f2 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:24.986652 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:46:24.996810 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 959/1919] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:46:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:46:25.013839 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3b6406-de46-4483-9a7b-83b23f34b6c7 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:25.024522 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3b6406-de46-4483-9a7b-83b23f34b6c7 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:25.029215 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bcb5ff-10ba-4e5d-a744-15ddcab550dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:25.029920 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bcb5ff-10ba-4e5d-a744-15ddcab550dc admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:25.030215 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bcb5ff-10ba-4e5d-a744-15ddcab550dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:25.030496 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bcb5ff-10ba-4e5d-a744-15ddcab550dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:25.031323 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bcb5ff-10ba-4e5d-a744-15ddcab550dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:25.033007 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bcb5ff-10ba-4e5d-a744-15ddcab550dc admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:25.046522 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 961/1920] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:46:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:46:25.050818 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10adffb8-4f2d-43e0-8932-208a408c51f2 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:25.061974 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10adffb8-4f2d-43e0-8932-208a408c51f2 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:25.066604 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c797a1d5-d1a9-49ba-9ff0-7721e0b57365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:25.067596 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c797a1d5-d1a9-49ba-9ff0-7721e0b57365 admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:25.067887 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c797a1d5-d1a9-49ba-9ff0-7721e0b57365 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:25.068165 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c797a1d5-d1a9-49ba-9ff0-7721e0b57365 admin admin] PATH_INFO: `/v3/projects/d01fea1c4ee8462cb4063acab062a92f/users/646fc1dc5b99405ca6d1c75eff3fb19c/roles/559942c4de1744169f8b5e0404ecde6b` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:25.083571 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c797a1d5-d1a9-49ba-9ff0-7721e0b57365 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d01fea1c4ee8462cb4063acab062a92f, user_id=646fc1dc5b99405ca6d1c75eff3fb19c, role_id=559942c4de1744169f8b5e0404ecde6b)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:25.086292 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c797a1d5-d1a9-49ba-9ff0-7721e0b57365 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:25.179088 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 960/1921] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:46:25 2022] PUT /identity/v3/projects/d01fea1c4ee8462cb4063acab062a92f/users/646fc1dc5b99405ca6d1c75eff3fb19c/roles/559942c4de1744169f8b5e0404ecde6b => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 22:46:25.195471 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243702df-d015-4ce6-baab-c1aef1fdc64e None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:25.195766 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243702df-d015-4ce6-baab-c1aef1fdc64e None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:25.196048 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243702df-d015-4ce6-baab-c1aef1fdc64e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:25.221549 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-243702df-d015-4ce6-baab-c1aef1fdc64e None None] MFA Rules not processed for user `646fc1dc5b99405ca6d1c75eff3fb19c`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:46:25.222695 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-243702df-d015-4ce6-baab-c1aef1fdc64e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:46:25.255042 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:46:25.287580 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 962/1922] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:46:25 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:46:25.292444 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c797a1d5-d1a9-49ba-9ff0-7721e0b57365 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:25.353267 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c797a1d5-d1a9-49ba-9ff0-7721e0b57365 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:25.358425 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2265eb3f-f91a-4622-b8c2-fb738ddf6be2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:25.359114 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2265eb3f-f91a-4622-b8c2-fb738ddf6be2 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:25.359406 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2265eb3f-f91a-4622-b8c2-fb738ddf6be2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:25.359688 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2265eb3f-f91a-4622-b8c2-fb738ddf6be2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:25.360377 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2265eb3f-f91a-4622-b8c2-fb738ddf6be2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:25.362292 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2265eb3f-f91a-4622-b8c2-fb738ddf6be2 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:25.448566 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 961/1923] 10.222.0.81 () {60 vars in 1152 bytes} [Wed Jul 20 22:46:25 2022] POST /identity/v3/projects => generated 399 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:46:25.452802 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243702df-d015-4ce6-baab-c1aef1fdc64e None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:25.463653 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243702df-d015-4ce6-baab-c1aef1fdc64e None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:25.522766 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bf8619-8f18-4acf-912f-8378983c9ac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:25.523512 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bf8619-8f18-4acf-912f-8378983c9ac6 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:25.523799 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bf8619-8f18-4acf-912f-8378983c9ac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:25.524077 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bf8619-8f18-4acf-912f-8378983c9ac6 admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:25.524783 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bf8619-8f18-4acf-912f-8378983c9ac6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:25.526706 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bf8619-8f18-4acf-912f-8378983c9ac6 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:25.650498 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 963/1924] 10.222.0.81 () {60 vars in 1143 bytes} [Wed Jul 20 22:46:25 2022] POST /identity/v3/users => generated 306 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 22:46:25.651730 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2265eb3f-f91a-4622-b8c2-fb738ddf6be2 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:25.662580 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2265eb3f-f91a-4622-b8c2-fb738ddf6be2 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:25.667756 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91a5279-4556-45c4-a831-7fb3c9e801d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:25.668425 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91a5279-4556-45c4-a831-7fb3c9e801d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:25.668717 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91a5279-4556-45c4-a831-7fb3c9e801d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:25.668998 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91a5279-4556-45c4-a831-7fb3c9e801d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:25.671573 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91a5279-4556-45c4-a831-7fb3c9e801d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:25.671986 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91a5279-4556-45c4-a831-7fb3c9e801d6 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:25.683002 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 962/1925] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:46:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:46:25.691074 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bf8619-8f18-4acf-912f-8378983c9ac6 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:25.701132 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bf8619-8f18-4acf-912f-8378983c9ac6 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:25.706038 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c07aba5-e090-4be2-adb4-46def564be62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:25.706948 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c07aba5-e090-4be2-adb4-46def564be62 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:25.707267 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c07aba5-e090-4be2-adb4-46def564be62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:25.707551 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c07aba5-e090-4be2-adb4-46def564be62 admin admin] PATH_INFO: `/v3/projects/d807ad639d5143d9b7a2867505927b01/users/b605483d42fd4ea3b6b47ac7e8a10d65/roles/d89b6c92d35a401a8517f1d47f6be843` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:25.723774 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c07aba5-e090-4be2-adb4-46def564be62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d807ad639d5143d9b7a2867505927b01, user_id=b605483d42fd4ea3b6b47ac7e8a10d65, role_id=d89b6c92d35a401a8517f1d47f6be843)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:25.726412 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c07aba5-e090-4be2-adb4-46def564be62 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:25.802740 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 964/1926] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:46:25 2022] PUT /identity/v3/projects/d807ad639d5143d9b7a2867505927b01/users/b605483d42fd4ea3b6b47ac7e8a10d65/roles/d89b6c92d35a401a8517f1d47f6be843 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:46:25.806821 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91a5279-4556-45c4-a831-7fb3c9e801d6 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:25.817163 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91a5279-4556-45c4-a831-7fb3c9e801d6 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:25.883837 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2103d7-1f01-4016-a897-8e1ebbada9ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:25.884540 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2103d7-1f01-4016-a897-8e1ebbada9ea admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:25.890030 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2103d7-1f01-4016-a897-8e1ebbada9ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:25.890030 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2103d7-1f01-4016-a897-8e1ebbada9ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:25.890030 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2103d7-1f01-4016-a897-8e1ebbada9ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:25.891452 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2103d7-1f01-4016-a897-8e1ebbada9ea admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:25.904629 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 963/1927] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:46:25 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:46:25.909642 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c07aba5-e090-4be2-adb4-46def564be62 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:25.919883 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c07aba5-e090-4be2-adb4-46def564be62 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:25.924541 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50d164f-8987-49eb-b736-27602623ae95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:25.925434 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50d164f-8987-49eb-b736-27602623ae95 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:25.925769 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50d164f-8987-49eb-b736-27602623ae95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:25.926083 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50d164f-8987-49eb-b736-27602623ae95 admin admin] PATH_INFO: `/v3/projects/d807ad639d5143d9b7a2867505927b01/users/b605483d42fd4ea3b6b47ac7e8a10d65/roles/d89b6c92d35a401a8517f1d47f6be843` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:25.930047 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50d164f-8987-49eb-b736-27602623ae95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d807ad639d5143d9b7a2867505927b01, user_id=b605483d42fd4ea3b6b47ac7e8a10d65, role_id=d89b6c92d35a401a8517f1d47f6be843)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:25.930720 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50d164f-8987-49eb-b736-27602623ae95 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:25.941261 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 965/1928] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:46:25 2022] PUT /identity/v3/projects/d807ad639d5143d9b7a2867505927b01/users/b605483d42fd4ea3b6b47ac7e8a10d65/roles/d89b6c92d35a401a8517f1d47f6be843 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:46:25.952899 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a4e2fa-99f6-4272-89f2-18251e80b40e None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:25.953197 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a4e2fa-99f6-4272-89f2-18251e80b40e None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:25.953487 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a4e2fa-99f6-4272-89f2-18251e80b40e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:25.980199 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-66a4e2fa-99f6-4272-89f2-18251e80b40e None None] MFA Rules not processed for user `b605483d42fd4ea3b6b47ac7e8a10d65`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:46:25.981337 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-66a4e2fa-99f6-4272-89f2-18251e80b40e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:46:26.010896 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:46:26.052713 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 964/1929] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:46:25 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:46:26.064509 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50d164f-8987-49eb-b736-27602623ae95 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:26.066594 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-d50d164f-8987-49eb-b736-27602623ae95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:46:26.124532 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50d164f-8987-49eb-b736-27602623ae95 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:26.128333 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b58a530-117b-440a-8ae6-dcced8964140 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecf4db289fba4e4a806198c62d31f7c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:26.129027 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b58a530-117b-440a-8ae6-dcced8964140 service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:26.129332 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b58a530-117b-440a-8ae6-dcced8964140 service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:26.129599 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b58a530-117b-440a-8ae6-dcced8964140 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:26.139906 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b58a530-117b-440a-8ae6-dcced8964140 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:26.141355 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b58a530-117b-440a-8ae6-dcced8964140 service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:26.152743 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 966/1930] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:46:26 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:46:26.494951 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a4e2fa-99f6-4272-89f2-18251e80b40e None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:26.496382 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-66a4e2fa-99f6-4272-89f2-18251e80b40e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:46:26.509222 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a4e2fa-99f6-4272-89f2-18251e80b40e None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:26.514060 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230f79bb-9844-42ce-90c0-8ab9fa8dec0f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecf4db289fba4e4a806198c62d31f7c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:26.514060 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230f79bb-9844-42ce-90c0-8ab9fa8dec0f service cinder] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:26.514060 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230f79bb-9844-42ce-90c0-8ab9fa8dec0f service cinder] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:26.514060 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230f79bb-9844-42ce-90c0-8ab9fa8dec0f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:26.575109 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230f79bb-9844-42ce-90c0-8ab9fa8dec0f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:26.576114 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230f79bb-9844-42ce-90c0-8ab9fa8dec0f service cinder] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:26.585158 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 965/1931] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:46:26 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 22:46:36.148955 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=82,keystone:INSERT=5 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:46:36.580974 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=43 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:46:45.021996 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b58a530-117b-440a-8ae6-dcced8964140 service cinder] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:45.023897 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-9b58a530-117b-440a-8ae6-dcced8964140 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:46:45.102205 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b58a530-117b-440a-8ae6-dcced8964140 service cinder] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:45.107833 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e00a88-565b-4d03-af3c-f753ab09364d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:45.108777 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e00a88-565b-4d03-af3c-f753ab09364d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:45.109070 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e00a88-565b-4d03-af3c-f753ab09364d admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:45.109353 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e00a88-565b-4d03-af3c-f753ab09364d admin admin] PATH_INFO: `/v3/users/c05c4f58865c491b81dab0e5af07ce4f` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:45.110578 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e00a88-565b-4d03-af3c-f753ab09364d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c05c4f58865c491b81dab0e5af07ce4f)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:45.112533 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e00a88-565b-4d03-af3c-f753ab09364d admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:45.569706 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-27e00a88-565b-4d03-af3c-f753ab09364d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c05c4f58865c491b81dab0e5af07ce4f'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:45.570448 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-27e00a88-565b-4d03-af3c-f753ab09364d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c05c4f58865c491b81dab0e5af07ce4f'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:45.571141 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-27e00a88-565b-4d03-af3c-f753ab09364d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c05c4f58865c491b81dab0e5af07ce4f'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:45.579135 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-27e00a88-565b-4d03-af3c-f753ab09364d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c05c4f58865c491b81dab0e5af07ce4f'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:45.773823 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-27e00a88-565b-4d03-af3c-f753ab09364d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c05c4f58865c491b81dab0e5af07ce4f'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:45.793084 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 967/1932] 10.222.0.81 () {58 vars in 1223 bytes} [Wed Jul 20 22:46:45 2022] DELETE /identity/v3/users/c05c4f58865c491b81dab0e5af07ce4f => generated 0 bytes in 772 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:46:45.797624 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ebd184-2296-4edf-bf5d-1707eb0a0657 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:45.798449 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ebd184-2296-4edf-bf5d-1707eb0a0657 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:45.798950 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ebd184-2296-4edf-bf5d-1707eb0a0657 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:45.817559 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-c7ebd184-2296-4edf-bf5d-1707eb0a0657 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:46:45.821582 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-c7ebd184-2296-4edf-bf5d-1707eb0a0657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:46:45.880899 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 966/1933] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:46:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:46:45.891753 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e00a88-565b-4d03-af3c-f753ab09364d admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:45.893167 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-27e00a88-565b-4d03-af3c-f753ab09364d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:46:45.951817 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e00a88-565b-4d03-af3c-f753ab09364d admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:45.955626 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4127cba4-c705-4b73-b542-d9014a11d8d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:45.956293 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4127cba4-c705-4b73-b542-d9014a11d8d1 service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:45.956585 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4127cba4-c705-4b73-b542-d9014a11d8d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:45.956899 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4127cba4-c705-4b73-b542-d9014a11d8d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:45.967038 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ebd184-2296-4edf-bf5d-1707eb0a0657 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:45.968447 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-c7ebd184-2296-4edf-bf5d-1707eb0a0657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:46:45.969919 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4127cba4-c705-4b73-b542-d9014a11d8d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:45.971689 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4127cba4-c705-4b73-b542-d9014a11d8d1 service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:45.986954 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 968/1934] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:46:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:46:45.992489 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ebd184-2296-4edf-bf5d-1707eb0a0657 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:45.998067 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8281508a-c7bd-46f1-b0e1-a019cf6316a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:45.999581 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8281508a-c7bd-46f1-b0e1-a019cf6316a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:45.999870 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8281508a-c7bd-46f1-b0e1-a019cf6316a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:46.000153 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8281508a-c7bd-46f1-b0e1-a019cf6316a4 admin admin] PATH_INFO: `/v3/users/646fc1dc5b99405ca6d1c75eff3fb19c` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:46.001410 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8281508a-c7bd-46f1-b0e1-a019cf6316a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=646fc1dc5b99405ca6d1c75eff3fb19c)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:46.003864 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8281508a-c7bd-46f1-b0e1-a019cf6316a4 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:46.345468 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-8281508a-c7bd-46f1-b0e1-a019cf6316a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '646fc1dc5b99405ca6d1c75eff3fb19c'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:46.346130 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-8281508a-c7bd-46f1-b0e1-a019cf6316a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '646fc1dc5b99405ca6d1c75eff3fb19c'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:46.346752 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-8281508a-c7bd-46f1-b0e1-a019cf6316a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '646fc1dc5b99405ca6d1c75eff3fb19c'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:46.353881 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-8281508a-c7bd-46f1-b0e1-a019cf6316a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '646fc1dc5b99405ca6d1c75eff3fb19c'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:46.463912 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-8281508a-c7bd-46f1-b0e1-a019cf6316a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '646fc1dc5b99405ca6d1c75eff3fb19c'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:46.483918 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 967/1935] 10.222.0.81 () {58 vars in 1223 bytes} [Wed Jul 20 22:46:45 2022] DELETE /identity/v3/users/646fc1dc5b99405ca6d1c75eff3fb19c => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:46:46.493120 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47baeb7-f57d-48ef-b4aa-0ec52f25513a None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:46.493632 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47baeb7-f57d-48ef-b4aa-0ec52f25513a None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:46.493725 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47baeb7-f57d-48ef-b4aa-0ec52f25513a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:46.510784 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-b47baeb7-f57d-48ef-b4aa-0ec52f25513a None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:46:46.512020 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-b47baeb7-f57d-48ef-b4aa-0ec52f25513a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:46:46.528687 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:46:46.562701 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8281508a-c7bd-46f1-b0e1-a019cf6316a4 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:46.564207 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-8281508a-c7bd-46f1-b0e1-a019cf6316a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:46:46.571470 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 969/1936] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:46:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:46:46.582788 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47baeb7-f57d-48ef-b4aa-0ec52f25513a None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:46.584508 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-b47baeb7-f57d-48ef-b4aa-0ec52f25513a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:46:46.585210 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8281508a-c7bd-46f1-b0e1-a019cf6316a4 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:46.590517 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9fee11-d754-4b8a-b2ec-44b1ab6296e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:46.591974 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9fee11-d754-4b8a-b2ec-44b1ab6296e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:46.592263 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9fee11-d754-4b8a-b2ec-44b1ab6296e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:46.592536 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9fee11-d754-4b8a-b2ec-44b1ab6296e3 admin admin] PATH_INFO: `/v3/projects/a243aa5fdb324a67bcd1d7e8b78ed3d1` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:46.593765 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9fee11-d754-4b8a-b2ec-44b1ab6296e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a243aa5fdb324a67bcd1d7e8b78ed3d1)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:46.595749 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9fee11-d754-4b8a-b2ec-44b1ab6296e3 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:46.644919 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47baeb7-f57d-48ef-b4aa-0ec52f25513a None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:46.648693 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbc0a18-5c5a-49cf-8773-c374cd8587cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:46.649360 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbc0a18-5c5a-49cf-8773-c374cd8587cd service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:46.649650 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbc0a18-5c5a-49cf-8773-c374cd8587cd service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:46.649936 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbc0a18-5c5a-49cf-8773-c374cd8587cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:46.659954 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbc0a18-5c5a-49cf-8773-c374cd8587cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:46.661377 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbc0a18-5c5a-49cf-8773-c374cd8587cd service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:46.672426 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 970/1937] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:46:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 22:46:46.717150 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-fb9fee11-d754-4b8a-b2ec-44b1ab6296e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a243aa5fdb324a67bcd1d7e8b78ed3d1', 'request_id': 'req-fb9fee11-d754-4b8a-b2ec-44b1ab6296e3'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:46.789121 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-fb9fee11-d754-4b8a-b2ec-44b1ab6296e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a243aa5fdb324a67bcd1d7e8b78ed3d1', 'request_id': 'req-fb9fee11-d754-4b8a-b2ec-44b1ab6296e3'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:46.804450 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-fb9fee11-d754-4b8a-b2ec-44b1ab6296e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a243aa5fdb324a67bcd1d7e8b78ed3d1', 'request_id': 'req-fb9fee11-d754-4b8a-b2ec-44b1ab6296e3'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:46.805673 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:46:46.818470 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-fb9fee11-d754-4b8a-b2ec-44b1ab6296e3 admin admin] The token cache is being invalidate because project a243aa5fdb324a67bcd1d7e8b78ed3d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:46:46.818774 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-fb9fee11-d754-4b8a-b2ec-44b1ab6296e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:46.819497 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-fb9fee11-d754-4b8a-b2ec-44b1ab6296e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:46.824465 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 968/1938] 10.222.0.81 () {58 vars in 1232 bytes} [Wed Jul 20 22:46:46 2022] DELETE /identity/v3/projects/a243aa5fdb324a67bcd1d7e8b78ed3d1 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 22:46:46.825297 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbc0a18-5c5a-49cf-8773-c374cd8587cd service neutron] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:46.836973 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbc0a18-5c5a-49cf-8773-c374cd8587cd service neutron] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:46.909794 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4f4e3f-35fd-4047-be68-dc087b89bc6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:46.910723 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4f4e3f-35fd-4047-be68-dc087b89bc6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:46.911062 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4f4e3f-35fd-4047-be68-dc087b89bc6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:46.911342 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4f4e3f-35fd-4047-be68-dc087b89bc6b admin admin] PATH_INFO: `/v3/users/abdb0384039b4e449e5f7ae2b45cca47` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:46.912588 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4f4e3f-35fd-4047-be68-dc087b89bc6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=abdb0384039b4e449e5f7ae2b45cca47)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:46.914535 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4f4e3f-35fd-4047-be68-dc087b89bc6b admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:47.146967 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-fd4f4e3f-35fd-4047-be68-dc087b89bc6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abdb0384039b4e449e5f7ae2b45cca47'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:47.147744 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-fd4f4e3f-35fd-4047-be68-dc087b89bc6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abdb0384039b4e449e5f7ae2b45cca47'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:47.148379 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-fd4f4e3f-35fd-4047-be68-dc087b89bc6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abdb0384039b4e449e5f7ae2b45cca47'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:47.155140 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-fd4f4e3f-35fd-4047-be68-dc087b89bc6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abdb0384039b4e449e5f7ae2b45cca47'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:47.169870 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9fee11-d754-4b8a-b2ec-44b1ab6296e3 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:47.171516 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-fb9fee11-d754-4b8a-b2ec-44b1ab6296e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:46:47.188464 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9fee11-d754-4b8a-b2ec-44b1ab6296e3 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:47.205317 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec0b244-77be-4354-a122-7aa4ed144d69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:47.206256 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec0b244-77be-4354-a122-7aa4ed144d69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:47.206545 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec0b244-77be-4354-a122-7aa4ed144d69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:47.206825 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec0b244-77be-4354-a122-7aa4ed144d69 admin admin] PATH_INFO: `/v3/projects/d01fea1c4ee8462cb4063acab062a92f` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:47.208553 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec0b244-77be-4354-a122-7aa4ed144d69 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d01fea1c4ee8462cb4063acab062a92f)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:47.210482 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec0b244-77be-4354-a122-7aa4ed144d69 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:47.298074 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-fd4f4e3f-35fd-4047-be68-dc087b89bc6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abdb0384039b4e449e5f7ae2b45cca47'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:47.305679 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:46:47.329731 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 971/1939] 10.222.0.81 () {58 vars in 1223 bytes} [Wed Jul 20 22:46:46 2022] DELETE /identity/v3/users/abdb0384039b4e449e5f7ae2b45cca47 => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:46:47.385212 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-dec0b244-77be-4354-a122-7aa4ed144d69 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd01fea1c4ee8462cb4063acab062a92f', 'request_id': 'req-dec0b244-77be-4354-a122-7aa4ed144d69'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:47.486569 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-dec0b244-77be-4354-a122-7aa4ed144d69 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd01fea1c4ee8462cb4063acab062a92f', 'request_id': 'req-dec0b244-77be-4354-a122-7aa4ed144d69'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:47.494879 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-dec0b244-77be-4354-a122-7aa4ed144d69 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd01fea1c4ee8462cb4063acab062a92f', 'request_id': 'req-dec0b244-77be-4354-a122-7aa4ed144d69'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:47.507300 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-dec0b244-77be-4354-a122-7aa4ed144d69 admin admin] The token cache is being invalidate because project d01fea1c4ee8462cb4063acab062a92f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:46:47.507593 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-dec0b244-77be-4354-a122-7aa4ed144d69 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:47.508587 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-dec0b244-77be-4354-a122-7aa4ed144d69 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:47.510282 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 969/1940] 10.222.0.81 () {58 vars in 1232 bytes} [Wed Jul 20 22:46:47 2022] DELETE /identity/v3/projects/d01fea1c4ee8462cb4063acab062a92f => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 22:46:47.514777 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4f4e3f-35fd-4047-be68-dc087b89bc6b admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:47.516580 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-fd4f4e3f-35fd-4047-be68-dc087b89bc6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:46:47.582450 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4f4e3f-35fd-4047-be68-dc087b89bc6b admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:47.636062 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a95926-a391-46e4-998f-95131f250c5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:47.637003 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a95926-a391-46e4-998f-95131f250c5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:47.637322 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a95926-a391-46e4-998f-95131f250c5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:47.637602 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a95926-a391-46e4-998f-95131f250c5c admin admin] PATH_INFO: `/v3/users/b605483d42fd4ea3b6b47ac7e8a10d65` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:47.639367 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a95926-a391-46e4-998f-95131f250c5c admin admin] RBAC: Authorizing `identity:delete_user(user_id=b605483d42fd4ea3b6b47ac7e8a10d65)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:47.641367 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a95926-a391-46e4-998f-95131f250c5c admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:47.873859 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-56a95926-a391-46e4-998f-95131f250c5c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b605483d42fd4ea3b6b47ac7e8a10d65'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:47.884261 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-56a95926-a391-46e4-998f-95131f250c5c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b605483d42fd4ea3b6b47ac7e8a10d65'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:47.885125 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-56a95926-a391-46e4-998f-95131f250c5c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b605483d42fd4ea3b6b47ac7e8a10d65'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:47.895548 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-56a95926-a391-46e4-998f-95131f250c5c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b605483d42fd4ea3b6b47ac7e8a10d65'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:48.029647 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-56a95926-a391-46e4-998f-95131f250c5c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b605483d42fd4ea3b6b47ac7e8a10d65'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:48.048155 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 972/1941] 10.222.0.81 () {58 vars in 1223 bytes} [Wed Jul 20 22:46:47 2022] DELETE /identity/v3/users/b605483d42fd4ea3b6b47ac7e8a10d65 => generated 0 bytes in 534 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 22:46:48.188641 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec0b244-77be-4354-a122-7aa4ed144d69 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:48.188641 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-dec0b244-77be-4354-a122-7aa4ed144d69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:46:48.215905 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec0b244-77be-4354-a122-7aa4ed144d69 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:48.279038 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49537a56-580d-429a-8db9-f27db658bd1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:48.279939 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49537a56-580d-429a-8db9-f27db658bd1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:48.280226 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49537a56-580d-429a-8db9-f27db658bd1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:48.280527 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49537a56-580d-429a-8db9-f27db658bd1f admin admin] PATH_INFO: `/v3/projects/0048e079b1bc420a88be6b02d09f599b` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:48.281690 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49537a56-580d-429a-8db9-f27db658bd1f admin admin] RBAC: Authorizing `identity:delete_project(project_id=0048e079b1bc420a88be6b02d09f599b)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:48.283895 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49537a56-580d-429a-8db9-f27db658bd1f admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:48.411365 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-49537a56-580d-429a-8db9-f27db658bd1f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0048e079b1bc420a88be6b02d09f599b', 'request_id': 'req-49537a56-580d-429a-8db9-f27db658bd1f'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:48.532769 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-49537a56-580d-429a-8db9-f27db658bd1f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0048e079b1bc420a88be6b02d09f599b', 'request_id': 'req-49537a56-580d-429a-8db9-f27db658bd1f'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:48.533606 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-49537a56-580d-429a-8db9-f27db658bd1f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0048e079b1bc420a88be6b02d09f599b', 'request_id': 'req-49537a56-580d-429a-8db9-f27db658bd1f'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:48.552385 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-49537a56-580d-429a-8db9-f27db658bd1f admin admin] The token cache is being invalidate because project 0048e079b1bc420a88be6b02d09f599b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:46:48.552685 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-49537a56-580d-429a-8db9-f27db658bd1f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:48.555204 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-49537a56-580d-429a-8db9-f27db658bd1f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:48.559830 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 970/1942] 10.222.0.81 () {58 vars in 1232 bytes} [Wed Jul 20 22:46:48 2022] DELETE /identity/v3/projects/0048e079b1bc420a88be6b02d09f599b => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:46:48.564148 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7c0df2-1d89-4202-9db9-d73d3acc45cf None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:48.564615 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7c0df2-1d89-4202-9db9-d73d3acc45cf None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:48.564708 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7c0df2-1d89-4202-9db9-d73d3acc45cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:48.582478 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-cb7c0df2-1d89-4202-9db9-d73d3acc45cf None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:46:48.583728 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-cb7c0df2-1d89-4202-9db9-d73d3acc45cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:46:48.590259 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:46:48.647138 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 973/1943] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:46:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:46:48.655446 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b17160-b9a2-477c-a47a-1b16e57c02e1 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:48.655740 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b17160-b9a2-477c-a47a-1b16e57c02e1 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:48.656021 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b17160-b9a2-477c-a47a-1b16e57c02e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:48.678185 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7c0df2-1d89-4202-9db9-d73d3acc45cf None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:48.681565 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-cb7c0df2-1d89-4202-9db9-d73d3acc45cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:46:48.682986 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-99b17160-b9a2-477c-a47a-1b16e57c02e1 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:46:48.685687 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-99b17160-b9a2-477c-a47a-1b16e57c02e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:46:48.699551 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 971/1944] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:46:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:46:48.705621 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b17160-b9a2-477c-a47a-1b16e57c02e1 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:48.712960 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:46:48.721202 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7c0df2-1d89-4202-9db9-d73d3acc45cf None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:48.729204 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b17160-b9a2-477c-a47a-1b16e57c02e1 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:48.737947 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e39fa2-7b02-489c-9efc-a18bee888045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:48.748421 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e39fa2-7b02-489c-9efc-a18bee888045 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:48.748421 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e39fa2-7b02-489c-9efc-a18bee888045 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:48.748421 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e39fa2-7b02-489c-9efc-a18bee888045 admin admin] PATH_INFO: `/v3/projects/d807ad639d5143d9b7a2867505927b01` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:48.748421 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e39fa2-7b02-489c-9efc-a18bee888045 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d807ad639d5143d9b7a2867505927b01)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:48.748421 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e39fa2-7b02-489c-9efc-a18bee888045 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:48.750450 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23dc422-7e57-429d-b0b4-ef938d0e2475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:48.751201 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23dc422-7e57-429d-b0b4-ef938d0e2475 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:48.751491 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23dc422-7e57-429d-b0b4-ef938d0e2475 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:48.751774 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23dc422-7e57-429d-b0b4-ef938d0e2475 admin admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:48.752578 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23dc422-7e57-429d-b0b4-ef938d0e2475 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:46:48.752893 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23dc422-7e57-429d-b0b4-ef938d0e2475 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:48.754530 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23dc422-7e57-429d-b0b4-ef938d0e2475 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:48.771051 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 972/1945] 10.222.0.81 () {58 vars in 1152 bytes} [Wed Jul 20 22:46:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:46:48.776063 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23dc422-7e57-429d-b0b4-ef938d0e2475 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:48.793660 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23dc422-7e57-429d-b0b4-ef938d0e2475 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:48.798535 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa9a13c-6615-46d4-8120-553d28c4cdfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:48.799085 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa9a13c-6615-46d4-8120-553d28c4cdfb admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:48.799376 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa9a13c-6615-46d4-8120-553d28c4cdfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:48.799664 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa9a13c-6615-46d4-8120-553d28c4cdfb admin admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:48.808533 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa9a13c-6615-46d4-8120-553d28c4cdfb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:48.808533 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa9a13c-6615-46d4-8120-553d28c4cdfb admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:48.876043 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-05e39fa2-7b02-489c-9efc-a18bee888045 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd807ad639d5143d9b7a2867505927b01', 'request_id': 'req-05e39fa2-7b02-489c-9efc-a18bee888045'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:48.882176 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-05e39fa2-7b02-489c-9efc-a18bee888045 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd807ad639d5143d9b7a2867505927b01', 'request_id': 'req-05e39fa2-7b02-489c-9efc-a18bee888045'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:48.917029 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 973/1946] 10.222.0.81 () {60 vars in 1152 bytes} [Wed Jul 20 22:46:48 2022] POST /identity/v3/projects => generated 375 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:46:48.921564 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa9a13c-6615-46d4-8120-553d28c4cdfb admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:48.981285 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa9a13c-6615-46d4-8120-553d28c4cdfb admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:48.986370 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d09a45-ebfb-4587-ab85-b90c7709a2e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:48.987074 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d09a45-ebfb-4587-ab85-b90c7709a2e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:48.987364 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d09a45-ebfb-4587-ab85-b90c7709a2e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:48.987640 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d09a45-ebfb-4587-ab85-b90c7709a2e3 admin admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:48.988328 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d09a45-ebfb-4587-ab85-b90c7709a2e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:48.990227 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d09a45-ebfb-4587-ab85-b90c7709a2e3 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:49.006356 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-05e39fa2-7b02-489c-9efc-a18bee888045 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd807ad639d5143d9b7a2867505927b01', 'request_id': 'req-05e39fa2-7b02-489c-9efc-a18bee888045'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:49.017109 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-05e39fa2-7b02-489c-9efc-a18bee888045 admin admin] The token cache is being invalidate because project d807ad639d5143d9b7a2867505927b01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:46:49.017397 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-05e39fa2-7b02-489c-9efc-a18bee888045 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:49.018099 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-05e39fa2-7b02-489c-9efc-a18bee888045 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:49.019689 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 974/1947] 10.222.0.81 () {58 vars in 1232 bytes} [Wed Jul 20 22:46:48 2022] DELETE /identity/v3/projects/d807ad639d5143d9b7a2867505927b01 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 22:46:49.024367 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927bd877-e560-47ef-a442-b6535a508996 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:49.024653 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927bd877-e560-47ef-a442-b6535a508996 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:49.024929 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927bd877-e560-47ef-a442-b6535a508996 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:49.042786 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-927bd877-e560-47ef-a442-b6535a508996 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:46:49.044014 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-927bd877-e560-47ef-a442-b6535a508996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:46:49.053656 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 975/1948] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:46:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 22:46:49.061968 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1331712a-c833-44d0-91fe-6cbca889a25f None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:49.062263 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1331712a-c833-44d0-91fe-6cbca889a25f None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:49.062544 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1331712a-c833-44d0-91fe-6cbca889a25f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:49.082266 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-1331712a-c833-44d0-91fe-6cbca889a25f None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:46:49.084546 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-1331712a-c833-44d0-91fe-6cbca889a25f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:46:49.100852 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 976/1949] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:46:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 22:46:49.102306 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1331712a-c833-44d0-91fe-6cbca889a25f None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:49.113410 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1331712a-c833-44d0-91fe-6cbca889a25f None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:49.118474 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4345f9b-e38b-4f48-8f98-b0828ee2755e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:49.119199 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4345f9b-e38b-4f48-8f98-b0828ee2755e admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:49.119481 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4345f9b-e38b-4f48-8f98-b0828ee2755e admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:49.119763 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4345f9b-e38b-4f48-8f98-b0828ee2755e admin admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:49.120498 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4345f9b-e38b-4f48-8f98-b0828ee2755e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:46:49.120809 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4345f9b-e38b-4f48-8f98-b0828ee2755e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:49.122472 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4345f9b-e38b-4f48-8f98-b0828ee2755e admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:49.140305 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 977/1950] 10.222.0.81 () {58 vars in 1152 bytes} [Wed Jul 20 22:46:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:46:49.141365 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4345f9b-e38b-4f48-8f98-b0828ee2755e admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:49.145956 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 974/1951] 10.222.0.81 () {60 vars in 1143 bytes} [Wed Jul 20 22:46:48 2022] POST /identity/v3/users => generated 288 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 22:46:49.150183 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d09a45-ebfb-4587-ab85-b90c7709a2e3 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:49.152045 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-f6d09a45-ebfb-4587-ab85-b90c7709a2e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:46:49.153859 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4345f9b-e38b-4f48-8f98-b0828ee2755e admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:49.159060 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbda4a9-ae2d-4cc6-895d-91eb1f7449c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:49.159751 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbda4a9-ae2d-4cc6-895d-91eb1f7449c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:49.160030 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbda4a9-ae2d-4cc6-895d-91eb1f7449c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:49.160309 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbda4a9-ae2d-4cc6-895d-91eb1f7449c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:49.161012 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbda4a9-ae2d-4cc6-895d-91eb1f7449c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:49.162951 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbda4a9-ae2d-4cc6-895d-91eb1f7449c0 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:49.169796 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d09a45-ebfb-4587-ab85-b90c7709a2e3 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:49.174725 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f2eb69-eb5b-4665-8c06-a3951a740752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:49.175451 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f2eb69-eb5b-4665-8c06-a3951a740752 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:49.176092 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f2eb69-eb5b-4665-8c06-a3951a740752 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:49.176092 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f2eb69-eb5b-4665-8c06-a3951a740752 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:49.176985 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f2eb69-eb5b-4665-8c06-a3951a740752 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:49.183378 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f2eb69-eb5b-4665-8c06-a3951a740752 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:49.196421 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 975/1952] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:46:49 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 22:46:49.197445 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f2eb69-eb5b-4665-8c06-a3951a740752 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:49.207964 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f2eb69-eb5b-4665-8c06-a3951a740752 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:49.212836 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89886d3a-4525-477d-b436-939f33c8bada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:49.213580 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89886d3a-4525-477d-b436-939f33c8bada admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:49.213935 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89886d3a-4525-477d-b436-939f33c8bada admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:49.214284 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89886d3a-4525-477d-b436-939f33c8bada admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:49.216188 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89886d3a-4525-477d-b436-939f33c8bada admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:49.217048 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89886d3a-4525-477d-b436-939f33c8bada admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:49.230777 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 976/1953] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:46:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:46:49.235989 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89886d3a-4525-477d-b436-939f33c8bada admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:49.245970 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89886d3a-4525-477d-b436-939f33c8bada admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:49.257416 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c990964-a91c-4f64-850e-d111af01ac19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:49.258318 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c990964-a91c-4f64-850e-d111af01ac19 admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:49.264240 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c990964-a91c-4f64-850e-d111af01ac19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:49.264240 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c990964-a91c-4f64-850e-d111af01ac19 admin admin] PATH_INFO: `/v3/projects/8e1af1488f394f1fac3ce7665da98cda/users/25255394021f4896acae48c9f2ee0c31/roles/559942c4de1744169f8b5e0404ecde6b` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:49.273428 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c990964-a91c-4f64-850e-d111af01ac19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e1af1488f394f1fac3ce7665da98cda, user_id=25255394021f4896acae48c9f2ee0c31, role_id=559942c4de1744169f8b5e0404ecde6b)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:49.276130 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c990964-a91c-4f64-850e-d111af01ac19 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:49.302117 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 978/1954] 10.222.0.81 () {60 vars in 1152 bytes} [Wed Jul 20 22:46:49 2022] POST /identity/v3/projects => generated 389 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:46:49.306919 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbda4a9-ae2d-4cc6-895d-91eb1f7449c0 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:49.361640 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 977/1955] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:46:49 2022] PUT /identity/v3/projects/8e1af1488f394f1fac3ce7665da98cda/users/25255394021f4896acae48c9f2ee0c31/roles/559942c4de1744169f8b5e0404ecde6b => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:46:49.368909 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbda4a9-ae2d-4cc6-895d-91eb1f7449c0 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:49.372562 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4fba15-f045-4145-a355-e3bb422ab2ea None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:49.372933 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4fba15-f045-4145-a355-e3bb422ab2ea None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:49.373289 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4fba15-f045-4145-a355-e3bb422ab2ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:49.398624 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-fb4fba15-f045-4145-a355-e3bb422ab2ea None None] MFA Rules not processed for user `25255394021f4896acae48c9f2ee0c31`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:46:49.400058 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-fb4fba15-f045-4145-a355-e3bb422ab2ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:46:49.426914 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3a6793-18f0-4cf2-91fe-ebdb8bfd9972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:49.427638 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3a6793-18f0-4cf2-91fe-ebdb8bfd9972 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:49.428078 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3a6793-18f0-4cf2-91fe-ebdb8bfd9972 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:49.428357 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3a6793-18f0-4cf2-91fe-ebdb8bfd9972 admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:49.429068 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3a6793-18f0-4cf2-91fe-ebdb8bfd9972 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:49.431044 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3a6793-18f0-4cf2-91fe-ebdb8bfd9972 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:49.465783 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 978/1956] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:46:49 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:46:49.477348 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4fba15-f045-4145-a355-e3bb422ab2ea None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:49.478683 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-fb4fba15-f045-4145-a355-e3bb422ab2ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:46:49.524829 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 979/1957] 10.222.0.81 () {60 vars in 1143 bytes} [Wed Jul 20 22:46:49 2022] POST /identity/v3/users => generated 295 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:46:49.525790 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3a6793-18f0-4cf2-91fe-ebdb8bfd9972 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:49.537557 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3a6793-18f0-4cf2-91fe-ebdb8bfd9972 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:49.544012 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab20915d-f396-4f91-b009-b89aa1092c7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:49.545367 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab20915d-f396-4f91-b009-b89aa1092c7b admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:49.545592 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab20915d-f396-4f91-b009-b89aa1092c7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:49.545872 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab20915d-f396-4f91-b009-b89aa1092c7b admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:49.546659 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab20915d-f396-4f91-b009-b89aa1092c7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:49.548277 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab20915d-f396-4f91-b009-b89aa1092c7b admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:49.559047 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 980/1958] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:46:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 22:46:49.562493 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4fba15-f045-4145-a355-e3bb422ab2ea None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:49.567141 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab20915d-f396-4f91-b009-b89aa1092c7b admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:49.567671 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1731b5d7-c1be-4af9-94fd-4d7f3c0832f0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c44640ea0014426a9523bb1c457778b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:49.567671 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1731b5d7-c1be-4af9-94fd-4d7f3c0832f0 service glance] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:49.567671 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1731b5d7-c1be-4af9-94fd-4d7f3c0832f0 service glance] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:49.567916 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1731b5d7-c1be-4af9-94fd-4d7f3c0832f0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:49.574666 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab20915d-f396-4f91-b009-b89aa1092c7b admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:49.576496 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1731b5d7-c1be-4af9-94fd-4d7f3c0832f0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:49.577928 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1731b5d7-c1be-4af9-94fd-4d7f3c0832f0 service glance] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:49.579577 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05380b03-7b46-457f-ad92-115317fabd9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:49.580244 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05380b03-7b46-457f-ad92-115317fabd9c admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:49.580534 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05380b03-7b46-457f-ad92-115317fabd9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:49.580818 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05380b03-7b46-457f-ad92-115317fabd9c admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:49.581537 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05380b03-7b46-457f-ad92-115317fabd9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:49.583396 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05380b03-7b46-457f-ad92-115317fabd9c admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:49.587453 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 979/1959] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:46:49 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 22:46:49.596931 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 981/1960] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:46:49 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 22:46:49.600801 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1731b5d7-c1be-4af9-94fd-4d7f3c0832f0 service glance] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:49.614292 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1731b5d7-c1be-4af9-94fd-4d7f3c0832f0 service glance] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:49.619148 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b52cf2-1aa7-48b1-ba8a-1fdece94baa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:49.620046 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b52cf2-1aa7-48b1-ba8a-1fdece94baa4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:49.620361 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b52cf2-1aa7-48b1-ba8a-1fdece94baa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:49.620645 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b52cf2-1aa7-48b1-ba8a-1fdece94baa4 admin admin] PATH_INFO: `/v3/projects/fae144dee4bb4b359b9c9a5743ee04cc/users/334285c1fb804db485b1106f789e6631/roles/559942c4de1744169f8b5e0404ecde6b` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:49.636327 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b52cf2-1aa7-48b1-ba8a-1fdece94baa4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fae144dee4bb4b359b9c9a5743ee04cc, user_id=334285c1fb804db485b1106f789e6631, role_id=559942c4de1744169f8b5e0404ecde6b)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:49.639690 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b52cf2-1aa7-48b1-ba8a-1fdece94baa4 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:49.662529 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:46:49.733465 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 980/1961] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:46:49 2022] PUT /identity/v3/projects/fae144dee4bb4b359b9c9a5743ee04cc/users/334285c1fb804db485b1106f789e6631/roles/559942c4de1744169f8b5e0404ecde6b => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:46:49.743839 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021c1a03-f617-4740-bbce-931271a011f4 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:49.744135 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021c1a03-f617-4740-bbce-931271a011f4 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:49.744240 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:46:49.750178 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021c1a03-f617-4740-bbce-931271a011f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:49.793996 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-021c1a03-f617-4740-bbce-931271a011f4 None None] MFA Rules not processed for user `334285c1fb804db485b1106f789e6631`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:46:49.795282 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-021c1a03-f617-4740-bbce-931271a011f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:46:49.894028 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 982/1962] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:46:49 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:46:49.898190 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b52cf2-1aa7-48b1-ba8a-1fdece94baa4 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:49.973212 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b52cf2-1aa7-48b1-ba8a-1fdece94baa4 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:49.977848 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b507df-8eee-45a6-a318-d7044ff369ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:49.978518 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b507df-8eee-45a6-a318-d7044ff369ab admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:49.978801 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b507df-8eee-45a6-a318-d7044ff369ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:49.979130 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b507df-8eee-45a6-a318-d7044ff369ab admin admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:49.979822 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b507df-8eee-45a6-a318-d7044ff369ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:49.981736 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b507df-8eee-45a6-a318-d7044ff369ab admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:50.116506 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 981/1963] 10.222.0.81 () {60 vars in 1152 bytes} [Wed Jul 20 22:46:49 2022] POST /identity/v3/projects => generated 387 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:46:50.120260 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021c1a03-f617-4740-bbce-931271a011f4 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:50.131404 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021c1a03-f617-4740-bbce-931271a011f4 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:50.178672 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:46:50.189368 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6d0a75-b4a2-4f2a-af95-d49ec14c6488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:50.190074 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6d0a75-b4a2-4f2a-af95-d49ec14c6488 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:50.190361 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6d0a75-b4a2-4f2a-af95-d49ec14c6488 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:50.190642 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6d0a75-b4a2-4f2a-af95-d49ec14c6488 admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:50.191393 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6d0a75-b4a2-4f2a-af95-d49ec14c6488 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:50.193395 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6d0a75-b4a2-4f2a-af95-d49ec14c6488 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:50.363634 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 983/1964] 10.222.0.81 () {60 vars in 1143 bytes} [Wed Jul 20 22:46:50 2022] POST /identity/v3/users => generated 300 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:46:50.364747 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b507df-8eee-45a6-a318-d7044ff369ab admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:50.375442 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b507df-8eee-45a6-a318-d7044ff369ab admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:50.379978 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f69cc2-5b0d-416e-88db-a9de03decca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:50.380649 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f69cc2-5b0d-416e-88db-a9de03decca2 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:50.380932 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f69cc2-5b0d-416e-88db-a9de03decca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:50.381212 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f69cc2-5b0d-416e-88db-a9de03decca2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:50.381944 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f69cc2-5b0d-416e-88db-a9de03decca2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:50.383627 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f69cc2-5b0d-416e-88db-a9de03decca2 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:50.394833 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 982/1965] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:46:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 22:46:50.399055 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6d0a75-b4a2-4f2a-af95-d49ec14c6488 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:50.410376 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6d0a75-b4a2-4f2a-af95-d49ec14c6488 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:50.416547 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daa3caf-6399-4652-8061-0663270040b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:50.419143 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daa3caf-6399-4652-8061-0663270040b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:50.419143 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daa3caf-6399-4652-8061-0663270040b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:50.419143 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daa3caf-6399-4652-8061-0663270040b9 admin admin] PATH_INFO: `/v3/projects/5bc46f4ffd1848a2bb0ea37a90250de9/users/ad5df67604024c0697527c0c6ca79cfa/roles/d89b6c92d35a401a8517f1d47f6be843` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:50.434184 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daa3caf-6399-4652-8061-0663270040b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bc46f4ffd1848a2bb0ea37a90250de9, user_id=ad5df67604024c0697527c0c6ca79cfa, role_id=d89b6c92d35a401a8517f1d47f6be843)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:50.437508 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daa3caf-6399-4652-8061-0663270040b9 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:50.545204 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 984/1966] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:46:50 2022] PUT /identity/v3/projects/5bc46f4ffd1848a2bb0ea37a90250de9/users/ad5df67604024c0697527c0c6ca79cfa/roles/d89b6c92d35a401a8517f1d47f6be843 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:46:50.553662 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f69cc2-5b0d-416e-88db-a9de03decca2 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:50.566517 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f69cc2-5b0d-416e-88db-a9de03decca2 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:50.607978 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:46:50.636537 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47394ba2-7aab-456b-a483-d844c616bf76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:50.637166 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47394ba2-7aab-456b-a483-d844c616bf76 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:50.637450 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47394ba2-7aab-456b-a483-d844c616bf76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:50.637726 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47394ba2-7aab-456b-a483-d844c616bf76 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:50.638441 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47394ba2-7aab-456b-a483-d844c616bf76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:50.640082 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47394ba2-7aab-456b-a483-d844c616bf76 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:50.649848 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 983/1967] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:46:50 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:46:50.653979 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daa3caf-6399-4652-8061-0663270040b9 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:50.664734 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daa3caf-6399-4652-8061-0663270040b9 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:50.670021 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15868b36-b4d5-4e3c-b789-3a0e7f5fb7f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:50.670900 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15868b36-b4d5-4e3c-b789-3a0e7f5fb7f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:50.671571 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15868b36-b4d5-4e3c-b789-3a0e7f5fb7f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:50.671571 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15868b36-b4d5-4e3c-b789-3a0e7f5fb7f1 admin admin] PATH_INFO: `/v3/projects/5bc46f4ffd1848a2bb0ea37a90250de9/users/ad5df67604024c0697527c0c6ca79cfa/roles/d89b6c92d35a401a8517f1d47f6be843` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:50.673562 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15868b36-b4d5-4e3c-b789-3a0e7f5fb7f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bc46f4ffd1848a2bb0ea37a90250de9, user_id=ad5df67604024c0697527c0c6ca79cfa, role_id=d89b6c92d35a401a8517f1d47f6be843)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:50.676357 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15868b36-b4d5-4e3c-b789-3a0e7f5fb7f1 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:50.687476 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 985/1968] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:46:50 2022] PUT /identity/v3/projects/5bc46f4ffd1848a2bb0ea37a90250de9/users/ad5df67604024c0697527c0c6ca79cfa/roles/d89b6c92d35a401a8517f1d47f6be843 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 22:46:50.698685 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4748feee-5298-40bb-b811-bdda5f8c5d86 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:50.698974 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4748feee-5298-40bb-b811-bdda5f8c5d86 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:50.699279 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4748feee-5298-40bb-b811-bdda5f8c5d86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:50.725516 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-4748feee-5298-40bb-b811-bdda5f8c5d86 None None] MFA Rules not processed for user `ad5df67604024c0697527c0c6ca79cfa`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:46:50.726730 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-4748feee-5298-40bb-b811-bdda5f8c5d86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:46:50.798305 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 984/1969] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:46:50 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:46:50.809997 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15868b36-b4d5-4e3c-b789-3a0e7f5fb7f1 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:50.812149 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-15868b36-b4d5-4e3c-b789-3a0e7f5fb7f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:46:50.872943 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15868b36-b4d5-4e3c-b789-3a0e7f5fb7f1 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:50.876691 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bed1f5-d14b-4a92-873d-d657fb7fa8bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecf4db289fba4e4a806198c62d31f7c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:50.877351 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bed1f5-d14b-4a92-873d-d657fb7fa8bd service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:50.877641 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bed1f5-d14b-4a92-873d-d657fb7fa8bd service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:50.879867 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bed1f5-d14b-4a92-873d-d657fb7fa8bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:50.887575 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bed1f5-d14b-4a92-873d-d657fb7fa8bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:50.888907 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bed1f5-d14b-4a92-873d-d657fb7fa8bd service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:50.898721 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 986/1970] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:46:50 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:46:51.035727 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4748feee-5298-40bb-b811-bdda5f8c5d86 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:51.106812 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4748feee-5298-40bb-b811-bdda5f8c5d86 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:51.117465 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c043c07a-bd0e-4a2b-8bf9-63d93439dc97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:51.118373 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c043c07a-bd0e-4a2b-8bf9-63d93439dc97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:51.118662 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c043c07a-bd0e-4a2b-8bf9-63d93439dc97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:51.118942 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c043c07a-bd0e-4a2b-8bf9-63d93439dc97 admin admin] PATH_INFO: `/v3/users/334285c1fb804db485b1106f789e6631` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:51.124301 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c043c07a-bd0e-4a2b-8bf9-63d93439dc97 admin admin] RBAC: Authorizing `identity:delete_user(user_id=334285c1fb804db485b1106f789e6631)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:51.126305 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c043c07a-bd0e-4a2b-8bf9-63d93439dc97 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:51.526041 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-c043c07a-bd0e-4a2b-8bf9-63d93439dc97 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '334285c1fb804db485b1106f789e6631'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:51.528267 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-c043c07a-bd0e-4a2b-8bf9-63d93439dc97 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '334285c1fb804db485b1106f789e6631'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:51.529952 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-c043c07a-bd0e-4a2b-8bf9-63d93439dc97 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '334285c1fb804db485b1106f789e6631'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:51.546440 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-c043c07a-bd0e-4a2b-8bf9-63d93439dc97 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '334285c1fb804db485b1106f789e6631'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:51.550092 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:46:51.777960 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-c043c07a-bd0e-4a2b-8bf9-63d93439dc97 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '334285c1fb804db485b1106f789e6631'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:51.798250 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 985/1971] 10.222.0.81 () {58 vars in 1223 bytes} [Wed Jul 20 22:46:51 2022] DELETE /identity/v3/users/334285c1fb804db485b1106f789e6631 => generated 0 bytes in 763 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:46:51.803045 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98320d29-23fc-4bb9-a68e-96f5d713a35f None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:51.803572 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98320d29-23fc-4bb9-a68e-96f5d713a35f None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:51.803711 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98320d29-23fc-4bb9-a68e-96f5d713a35f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:51.821570 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-98320d29-23fc-4bb9-a68e-96f5d713a35f None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:46:51.822799 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-98320d29-23fc-4bb9-a68e-96f5d713a35f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:46:51.885655 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 987/1972] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:46:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:46:51.896108 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c043c07a-bd0e-4a2b-8bf9-63d93439dc97 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:51.897518 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-c043c07a-bd0e-4a2b-8bf9-63d93439dc97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:46:51.955080 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c043c07a-bd0e-4a2b-8bf9-63d93439dc97 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:51.959038 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed18fcb0-d7f8-4bca-82a4-ec0ca6690b46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:51.959871 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed18fcb0-d7f8-4bca-82a4-ec0ca6690b46 service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:51.967332 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed18fcb0-d7f8-4bca-82a4-ec0ca6690b46 service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:51.967332 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed18fcb0-d7f8-4bca-82a4-ec0ca6690b46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:51.971082 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed18fcb0-d7f8-4bca-82a4-ec0ca6690b46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:51.972518 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed18fcb0-d7f8-4bca-82a4-ec0ca6690b46 service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:51.982156 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 986/1973] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:46:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:46:52.699832 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98320d29-23fc-4bb9-a68e-96f5d713a35f None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:52.701672 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-98320d29-23fc-4bb9-a68e-96f5d713a35f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:46:52.717466 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98320d29-23fc-4bb9-a68e-96f5d713a35f None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:52.722906 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851f206a-5a3e-40d8-87d3-17d262326c89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:52.723842 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851f206a-5a3e-40d8-87d3-17d262326c89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:52.724126 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851f206a-5a3e-40d8-87d3-17d262326c89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:52.724730 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851f206a-5a3e-40d8-87d3-17d262326c89 admin admin] PATH_INFO: `/v3/projects/fae144dee4bb4b359b9c9a5743ee04cc` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:52.726496 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851f206a-5a3e-40d8-87d3-17d262326c89 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fae144dee4bb4b359b9c9a5743ee04cc)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:52.728530 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851f206a-5a3e-40d8-87d3-17d262326c89 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:52.846742 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:46:52.853033 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-851f206a-5a3e-40d8-87d3-17d262326c89 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fae144dee4bb4b359b9c9a5743ee04cc', 'request_id': 'req-851f206a-5a3e-40d8-87d3-17d262326c89'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:52.860104 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-851f206a-5a3e-40d8-87d3-17d262326c89 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fae144dee4bb4b359b9c9a5743ee04cc', 'request_id': 'req-851f206a-5a3e-40d8-87d3-17d262326c89'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:53.018010 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-851f206a-5a3e-40d8-87d3-17d262326c89 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fae144dee4bb4b359b9c9a5743ee04cc', 'request_id': 'req-851f206a-5a3e-40d8-87d3-17d262326c89'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:53.029234 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-851f206a-5a3e-40d8-87d3-17d262326c89 admin admin] The token cache is being invalidate because project fae144dee4bb4b359b9c9a5743ee04cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:46:53.035027 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-851f206a-5a3e-40d8-87d3-17d262326c89 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:53.035027 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-851f206a-5a3e-40d8-87d3-17d262326c89 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:53.035027 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 988/1974] 10.222.0.81 () {58 vars in 1232 bytes} [Wed Jul 20 22:46:52 2022] DELETE /identity/v3/projects/fae144dee4bb4b359b9c9a5743ee04cc => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 22:46:53.036315 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed18fcb0-d7f8-4bca-82a4-ec0ca6690b46 service neutron] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:53.047219 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed18fcb0-d7f8-4bca-82a4-ec0ca6690b46 service neutron] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:53.104782 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea906a3e-ab20-444b-b4fe-dd6dfb476528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:53.105686 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea906a3e-ab20-444b-b4fe-dd6dfb476528 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:53.105970 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea906a3e-ab20-444b-b4fe-dd6dfb476528 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:53.106248 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea906a3e-ab20-444b-b4fe-dd6dfb476528 admin admin] PATH_INFO: `/v3/users/ad5df67604024c0697527c0c6ca79cfa` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:53.108648 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea906a3e-ab20-444b-b4fe-dd6dfb476528 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad5df67604024c0697527c0c6ca79cfa)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:53.111789 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea906a3e-ab20-444b-b4fe-dd6dfb476528 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:53.380852 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-ea906a3e-ab20-444b-b4fe-dd6dfb476528 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad5df67604024c0697527c0c6ca79cfa'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:53.381675 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-ea906a3e-ab20-444b-b4fe-dd6dfb476528 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad5df67604024c0697527c0c6ca79cfa'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:53.382268 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-ea906a3e-ab20-444b-b4fe-dd6dfb476528 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad5df67604024c0697527c0c6ca79cfa'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:53.389834 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-ea906a3e-ab20-444b-b4fe-dd6dfb476528 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad5df67604024c0697527c0c6ca79cfa'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:53.486182 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-ea906a3e-ab20-444b-b4fe-dd6dfb476528 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad5df67604024c0697527c0c6ca79cfa'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:53.500021 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:46:53.507130 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 987/1975] 10.222.0.81 () {58 vars in 1223 bytes} [Wed Jul 20 22:46:53 2022] DELETE /identity/v3/users/ad5df67604024c0697527c0c6ca79cfa => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:46:53.961710 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851f206a-5a3e-40d8-87d3-17d262326c89 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:53.966635 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-851f206a-5a3e-40d8-87d3-17d262326c89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:46:53.979859 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851f206a-5a3e-40d8-87d3-17d262326c89 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:54.063897 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e76a988-cbac-4ba6-b0ca-e771538cc7af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:54.064789 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e76a988-cbac-4ba6-b0ca-e771538cc7af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:54.065075 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e76a988-cbac-4ba6-b0ca-e771538cc7af admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:54.065360 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e76a988-cbac-4ba6-b0ca-e771538cc7af admin admin] PATH_INFO: `/v3/projects/5bc46f4ffd1848a2bb0ea37a90250de9` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:54.066797 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e76a988-cbac-4ba6-b0ca-e771538cc7af admin admin] RBAC: Authorizing `identity:delete_project(project_id=5bc46f4ffd1848a2bb0ea37a90250de9)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:54.069318 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e76a988-cbac-4ba6-b0ca-e771538cc7af admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:54.232030 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-7e76a988-cbac-4ba6-b0ca-e771538cc7af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5bc46f4ffd1848a2bb0ea37a90250de9', 'request_id': 'req-7e76a988-cbac-4ba6-b0ca-e771538cc7af'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:54.239938 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-7e76a988-cbac-4ba6-b0ca-e771538cc7af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5bc46f4ffd1848a2bb0ea37a90250de9', 'request_id': 'req-7e76a988-cbac-4ba6-b0ca-e771538cc7af'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:54.384331 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-7e76a988-cbac-4ba6-b0ca-e771538cc7af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5bc46f4ffd1848a2bb0ea37a90250de9', 'request_id': 'req-7e76a988-cbac-4ba6-b0ca-e771538cc7af'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:54.404068 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-7e76a988-cbac-4ba6-b0ca-e771538cc7af admin admin] The token cache is being invalidate because project 5bc46f4ffd1848a2bb0ea37a90250de9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:46:54.404483 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-7e76a988-cbac-4ba6-b0ca-e771538cc7af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:54.405280 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-7e76a988-cbac-4ba6-b0ca-e771538cc7af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:54.407853 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 989/1976] 10.222.0.81 () {58 vars in 1232 bytes} [Wed Jul 20 22:46:53 2022] DELETE /identity/v3/projects/5bc46f4ffd1848a2bb0ea37a90250de9 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:46:54.413447 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82630a3a-e12d-40ff-9c29-fdb9174e2de4 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:54.413946 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82630a3a-e12d-40ff-9c29-fdb9174e2de4 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:54.414212 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82630a3a-e12d-40ff-9c29-fdb9174e2de4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:54.451319 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-82630a3a-e12d-40ff-9c29-fdb9174e2de4 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:46:54.452567 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-82630a3a-e12d-40ff-9c29-fdb9174e2de4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:46:54.532784 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 988/1977] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:46:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:46:54.541162 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b20f5fb-6277-4280-b516-23dcff5e8c83 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:54.541565 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b20f5fb-6277-4280-b516-23dcff5e8c83 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:54.541947 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b20f5fb-6277-4280-b516-23dcff5e8c83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:54.564954 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-1b20f5fb-6277-4280-b516-23dcff5e8c83 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:46:54.567379 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-1b20f5fb-6277-4280-b516-23dcff5e8c83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:46:54.585686 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 990/1978] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:46:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 22:46:54.594460 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82630a3a-e12d-40ff-9c29-fdb9174e2de4 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:54.606034 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82630a3a-e12d-40ff-9c29-fdb9174e2de4 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:54.617333 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1e602b-6f40-4981-95fb-af844232be86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:54.617333 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1e602b-6f40-4981-95fb-af844232be86 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:54.617333 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1e602b-6f40-4981-95fb-af844232be86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:54.617333 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1e602b-6f40-4981-95fb-af844232be86 admin admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:54.617666 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1e602b-6f40-4981-95fb-af844232be86 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:46:54.617666 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1e602b-6f40-4981-95fb-af844232be86 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:54.617666 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1e602b-6f40-4981-95fb-af844232be86 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:54.636225 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 989/1979] 10.222.0.81 () {58 vars in 1152 bytes} [Wed Jul 20 22:46:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:46:54.636798 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b20f5fb-6277-4280-b516-23dcff5e8c83 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:54.647710 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b20f5fb-6277-4280-b516-23dcff5e8c83 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:54.662525 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65c0917-fb4d-4b5f-a562-ad858da23ccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:54.663637 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65c0917-fb4d-4b5f-a562-ad858da23ccf admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:54.664069 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65c0917-fb4d-4b5f-a562-ad858da23ccf admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:54.664575 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65c0917-fb4d-4b5f-a562-ad858da23ccf admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:54.665439 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65c0917-fb4d-4b5f-a562-ad858da23ccf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:54.667556 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65c0917-fb4d-4b5f-a562-ad858da23ccf admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:54.790866 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 991/1980] 10.222.0.81 () {60 vars in 1152 bytes} [Wed Jul 20 22:46:54 2022] POST /identity/v3/projects => generated 369 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 22:46:54.795310 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1e602b-6f40-4981-95fb-af844232be86 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:54.806801 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1e602b-6f40-4981-95fb-af844232be86 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:54.871537 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432ef760-f78e-47a1-8fdc-7f057ee27330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:54.871537 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432ef760-f78e-47a1-8fdc-7f057ee27330 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:54.871537 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432ef760-f78e-47a1-8fdc-7f057ee27330 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:54.871783 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432ef760-f78e-47a1-8fdc-7f057ee27330 admin admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:54.872342 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432ef760-f78e-47a1-8fdc-7f057ee27330 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:54.874868 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432ef760-f78e-47a1-8fdc-7f057ee27330 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:55.019856 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 990/1981] 10.222.0.81 () {60 vars in 1143 bytes} [Wed Jul 20 22:46:54 2022] POST /identity/v3/users => generated 285 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:46:55.023928 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65c0917-fb4d-4b5f-a562-ad858da23ccf admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:55.035180 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65c0917-fb4d-4b5f-a562-ad858da23ccf admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:55.039980 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b95cca-baa2-4400-913c-241107eff875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:55.040660 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b95cca-baa2-4400-913c-241107eff875 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:55.040951 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b95cca-baa2-4400-913c-241107eff875 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:55.041233 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b95cca-baa2-4400-913c-241107eff875 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:55.041961 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b95cca-baa2-4400-913c-241107eff875 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:55.043606 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b95cca-baa2-4400-913c-241107eff875 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:55.054330 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 992/1982] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:46:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:46:55.061466 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432ef760-f78e-47a1-8fdc-7f057ee27330 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:55.071630 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432ef760-f78e-47a1-8fdc-7f057ee27330 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:55.076923 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94d5837-511a-4c0e-b535-a6ce40c36378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:55.077585 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94d5837-511a-4c0e-b535-a6ce40c36378 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:55.077877 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94d5837-511a-4c0e-b535-a6ce40c36378 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:55.078164 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94d5837-511a-4c0e-b535-a6ce40c36378 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:55.078921 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94d5837-511a-4c0e-b535-a6ce40c36378 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:55.080565 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94d5837-511a-4c0e-b535-a6ce40c36378 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:55.092028 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 991/1983] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:46:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:46:55.101457 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b95cca-baa2-4400-913c-241107eff875 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:55.111539 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b95cca-baa2-4400-913c-241107eff875 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:55.116253 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a52ef3-1c92-4ea1-966a-ee9f76b6210c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:55.117327 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a52ef3-1c92-4ea1-966a-ee9f76b6210c admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:55.117824 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a52ef3-1c92-4ea1-966a-ee9f76b6210c admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:55.118154 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a52ef3-1c92-4ea1-966a-ee9f76b6210c admin admin] PATH_INFO: `/v3/projects/28ff2b258273493aa3028ba5a7213eb4/users/b7539f5312b84ab384fc1205518c11a6/roles/559942c4de1744169f8b5e0404ecde6b` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:55.140351 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a52ef3-1c92-4ea1-966a-ee9f76b6210c admin admin] RBAC: Authorizing `identity:create_grant(project_id=28ff2b258273493aa3028ba5a7213eb4, user_id=b7539f5312b84ab384fc1205518c11a6, role_id=559942c4de1744169f8b5e0404ecde6b)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:55.143158 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a52ef3-1c92-4ea1-966a-ee9f76b6210c admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:55.146276 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:46:55.246980 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 993/1984] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:46:55 2022] PUT /identity/v3/projects/28ff2b258273493aa3028ba5a7213eb4/users/b7539f5312b84ab384fc1205518c11a6/roles/559942c4de1744169f8b5e0404ecde6b => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:46:55.252551 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131fb249-c56a-46c5-93b4-514a84a985f2 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:55.252842 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131fb249-c56a-46c5-93b4-514a84a985f2 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:55.253126 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131fb249-c56a-46c5-93b4-514a84a985f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:55.276679 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-131fb249-c56a-46c5-93b4-514a84a985f2 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:46:55.277906 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-131fb249-c56a-46c5-93b4-514a84a985f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:46:55.340714 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 992/1985] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:46:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:46:55.350676 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a52ef3-1c92-4ea1-966a-ee9f76b6210c admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:55.352438 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-c6a52ef3-1c92-4ea1-966a-ee9f76b6210c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:46:55.410103 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a52ef3-1c92-4ea1-966a-ee9f76b6210c admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:55.413656 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc055a1-95fa-44a5-85cc-cc538691ba57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:55.414308 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc055a1-95fa-44a5-85cc-cc538691ba57 service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:55.414590 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc055a1-95fa-44a5-85cc-cc538691ba57 service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:55.414866 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc055a1-95fa-44a5-85cc-cc538691ba57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:55.424130 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc055a1-95fa-44a5-85cc-cc538691ba57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:55.425601 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc055a1-95fa-44a5-85cc-cc538691ba57 service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:55.436033 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 994/1986] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:46:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:46:57.014549 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdb6f33-62c0-46db-95ee-88fc07f7ddb7 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:57.014549 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdb6f33-62c0-46db-95ee-88fc07f7ddb7 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:57.015079 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdb6f33-62c0-46db-95ee-88fc07f7ddb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:57.020920 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:46:57.046635 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-bcdb6f33-62c0-46db-95ee-88fc07f7ddb7 None None] MFA Rules not processed for user `b7539f5312b84ab384fc1205518c11a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:46:57.048133 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-bcdb6f33-62c0-46db-95ee-88fc07f7ddb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:46:57.143311 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 993/1987] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:46:57 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:46:57.147692 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc055a1-95fa-44a5-85cc-cc538691ba57 service neutron] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:57.160520 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc055a1-95fa-44a5-85cc-cc538691ba57 service neutron] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:57.168768 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dc5d56-61b0-4149-8fb3-6da5962e2a9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:57.170919 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dc5d56-61b0-4149-8fb3-6da5962e2a9a admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:57.171321 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dc5d56-61b0-4149-8fb3-6da5962e2a9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:57.171828 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dc5d56-61b0-4149-8fb3-6da5962e2a9a admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:57.172636 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dc5d56-61b0-4149-8fb3-6da5962e2a9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:57.174654 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dc5d56-61b0-4149-8fb3-6da5962e2a9a admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:57.274811 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdb6f33-62c0-46db-95ee-88fc07f7ddb7 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:57.276347 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-bcdb6f33-62c0-46db-95ee-88fc07f7ddb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:46:57.292118 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdb6f33-62c0-46db-95ee-88fc07f7ddb7 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:57.298419 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96d96f0-7c35-42bf-a60e-e31ce640f281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:57.299439 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96d96f0-7c35-42bf-a60e-e31ce640f281 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:57.299732 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96d96f0-7c35-42bf-a60e-e31ce640f281 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:57.300015 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96d96f0-7c35-42bf-a60e-e31ce640f281 admin admin] PATH_INFO: `/v3/users/a21e40cfdac143088ed1c32082f51beb` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:57.301391 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96d96f0-7c35-42bf-a60e-e31ce640f281 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a21e40cfdac143088ed1c32082f51beb)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:57.303541 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96d96f0-7c35-42bf-a60e-e31ce640f281 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:57.331637 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 995/1988] 10.222.0.81 () {60 vars in 1152 bytes} [Wed Jul 20 22:46:57 2022] POST /identity/v3/projects => generated 371 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:46:57.335227 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dc5d56-61b0-4149-8fb3-6da5962e2a9a admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:57.414275 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dc5d56-61b0-4149-8fb3-6da5962e2a9a admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:57.423559 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744ee9e4-f914-45d1-8c83-30db8f097721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:57.424661 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744ee9e4-f914-45d1-8c83-30db8f097721 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:57.425134 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744ee9e4-f914-45d1-8c83-30db8f097721 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:57.426138 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744ee9e4-f914-45d1-8c83-30db8f097721 admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:57.427079 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744ee9e4-f914-45d1-8c83-30db8f097721 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:57.429251 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744ee9e4-f914-45d1-8c83-30db8f097721 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:57.572840 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 996/1989] 10.222.0.81 () {60 vars in 1143 bytes} [Wed Jul 20 22:46:57 2022] POST /identity/v3/users => generated 292 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:46:57.576827 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744ee9e4-f914-45d1-8c83-30db8f097721 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:57.590586 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744ee9e4-f914-45d1-8c83-30db8f097721 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:57.596385 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dead755-64c1-4c12-8ae2-5554cc9b093e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:57.597167 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dead755-64c1-4c12-8ae2-5554cc9b093e admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:57.597528 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dead755-64c1-4c12-8ae2-5554cc9b093e admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:57.597812 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dead755-64c1-4c12-8ae2-5554cc9b093e admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:57.598551 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dead755-64c1-4c12-8ae2-5554cc9b093e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:57.611246 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dead755-64c1-4c12-8ae2-5554cc9b093e admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:57.631651 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 997/1990] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:46:57 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:46:57.635980 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dead755-64c1-4c12-8ae2-5554cc9b093e admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:57.649234 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-a96d96f0-7c35-42bf-a60e-e31ce640f281 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a21e40cfdac143088ed1c32082f51beb'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:57.650012 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-a96d96f0-7c35-42bf-a60e-e31ce640f281 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a21e40cfdac143088ed1c32082f51beb'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:57.651181 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-a96d96f0-7c35-42bf-a60e-e31ce640f281 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a21e40cfdac143088ed1c32082f51beb'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:57.651764 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dead755-64c1-4c12-8ae2-5554cc9b093e admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:57.658852 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da78d52-f672-4499-8ca2-1008167cad41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:57.661076 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da78d52-f672-4499-8ca2-1008167cad41 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:57.662108 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da78d52-f672-4499-8ca2-1008167cad41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:57.662649 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da78d52-f672-4499-8ca2-1008167cad41 admin admin] PATH_INFO: `/v3/projects/09068fa01c6f4d3fb04164ffca5b7cee/users/548054880850403ea95cf8e8b63f9b4f/roles/d89b6c92d35a401a8517f1d47f6be843` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:57.686364 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da78d52-f672-4499-8ca2-1008167cad41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09068fa01c6f4d3fb04164ffca5b7cee, user_id=548054880850403ea95cf8e8b63f9b4f, role_id=d89b6c92d35a401a8517f1d47f6be843)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:57.691930 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da78d52-f672-4499-8ca2-1008167cad41 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:57.693285 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-a96d96f0-7c35-42bf-a60e-e31ce640f281 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a21e40cfdac143088ed1c32082f51beb'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:57.834833 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-a96d96f0-7c35-42bf-a60e-e31ce640f281 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a21e40cfdac143088ed1c32082f51beb'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:57.844153 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 998/1991] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:46:57 2022] PUT /identity/v3/projects/09068fa01c6f4d3fb04164ffca5b7cee/users/548054880850403ea95cf8e8b63f9b4f/roles/d89b6c92d35a401a8517f1d47f6be843 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:46:57.848347 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da78d52-f672-4499-8ca2-1008167cad41 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:57.851081 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-0da78d52-f672-4499-8ca2-1008167cad41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:46:57.861443 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 994/1992] 10.222.0.81 () {58 vars in 1223 bytes} [Wed Jul 20 22:46:57 2022] DELETE /identity/v3/users/a21e40cfdac143088ed1c32082f51beb => generated 0 bytes in 587 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:46:57.865767 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74755db2-ef2e-4a2c-9b8e-6def96faf1f4 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:57.866132 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74755db2-ef2e-4a2c-9b8e-6def96faf1f4 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:57.866979 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74755db2-ef2e-4a2c-9b8e-6def96faf1f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:57.887477 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-74755db2-ef2e-4a2c-9b8e-6def96faf1f4 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:46:57.889615 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-74755db2-ef2e-4a2c-9b8e-6def96faf1f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:46:57.904268 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:46:57.936403 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da78d52-f672-4499-8ca2-1008167cad41 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:57.988076 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 995/1993] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:46:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:46:58.037245 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f21b0fb-1d02-4c56-886c-4c3975adefde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:58.038929 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f21b0fb-1d02-4c56-886c-4c3975adefde admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:58.039312 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f21b0fb-1d02-4c56-886c-4c3975adefde admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:58.039659 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f21b0fb-1d02-4c56-886c-4c3975adefde admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:58.040625 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f21b0fb-1d02-4c56-886c-4c3975adefde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:58.043596 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f21b0fb-1d02-4c56-886c-4c3975adefde admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:58.053917 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74755db2-ef2e-4a2c-9b8e-6def96faf1f4 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:58.055844 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-74755db2-ef2e-4a2c-9b8e-6def96faf1f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:46:58.066603 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 999/1994] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:46:57 2022] GET /identity/v3/roles => generated 1420 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:46:58.069644 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:46:58.074571 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f21b0fb-1d02-4c56-886c-4c3975adefde admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:58.093472 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f21b0fb-1d02-4c56-886c-4c3975adefde admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:58.104350 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78f99a1-7976-4f1f-ba36-bf8c2ecae4e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:58.106066 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78f99a1-7976-4f1f-ba36-bf8c2ecae4e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:58.110921 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78f99a1-7976-4f1f-ba36-bf8c2ecae4e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:58.110921 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78f99a1-7976-4f1f-ba36-bf8c2ecae4e2 admin admin] PATH_INFO: `/v3/projects/09068fa01c6f4d3fb04164ffca5b7cee/users/548054880850403ea95cf8e8b63f9b4f/roles/d89b6c92d35a401a8517f1d47f6be843` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:58.110921 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78f99a1-7976-4f1f-ba36-bf8c2ecae4e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09068fa01c6f4d3fb04164ffca5b7cee, user_id=548054880850403ea95cf8e8b63f9b4f, role_id=d89b6c92d35a401a8517f1d47f6be843)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:58.115752 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78f99a1-7976-4f1f-ba36-bf8c2ecae4e2 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:58.139941 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1000/1995] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:46:58 2022] PUT /identity/v3/projects/09068fa01c6f4d3fb04164ffca5b7cee/users/548054880850403ea95cf8e8b63f9b4f/roles/d89b6c92d35a401a8517f1d47f6be843 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:46:58.166444 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74755db2-ef2e-4a2c-9b8e-6def96faf1f4 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:58.244937 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98b0cb1-fec7-462f-b010-222e47e1b8dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:58.246092 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98b0cb1-fec7-462f-b010-222e47e1b8dd service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:58.274569 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98b0cb1-fec7-462f-b010-222e47e1b8dd service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:58.274569 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98b0cb1-fec7-462f-b010-222e47e1b8dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:58.351290 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98b0cb1-fec7-462f-b010-222e47e1b8dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:58.352808 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98b0cb1-fec7-462f-b010-222e47e1b8dd service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:58.363051 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 996/1996] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:46:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 22:46:59.471285 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78f99a1-7976-4f1f-ba36-bf8c2ecae4e2 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:59.474311 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-d78f99a1-7976-4f1f-ba36-bf8c2ecae4e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:46:59.489825 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78f99a1-7976-4f1f-ba36-bf8c2ecae4e2 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:46:59.507767 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9503a2-d8a1-4d68-a564-08d67a5343d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:46:59.510592 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9503a2-d8a1-4d68-a564-08d67a5343d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:46:59.510839 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9503a2-d8a1-4d68-a564-08d67a5343d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:46:59.512093 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9503a2-d8a1-4d68-a564-08d67a5343d6 admin admin] PATH_INFO: `/v3/projects/2051ea10919f4922a6b237dbc996b6d1` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:46:59.514805 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9503a2-d8a1-4d68-a564-08d67a5343d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2051ea10919f4922a6b237dbc996b6d1)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:46:59.517033 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9503a2-d8a1-4d68-a564-08d67a5343d6 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:46:59.807361 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-6d9503a2-d8a1-4d68-a564-08d67a5343d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2051ea10919f4922a6b237dbc996b6d1', 'request_id': 'req-6d9503a2-d8a1-4d68-a564-08d67a5343d6'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:59.816567 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-6d9503a2-d8a1-4d68-a564-08d67a5343d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2051ea10919f4922a6b237dbc996b6d1', 'request_id': 'req-6d9503a2-d8a1-4d68-a564-08d67a5343d6'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:59.889363 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-6d9503a2-d8a1-4d68-a564-08d67a5343d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2051ea10919f4922a6b237dbc996b6d1', 'request_id': 'req-6d9503a2-d8a1-4d68-a564-08d67a5343d6'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:59.914713 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-6d9503a2-d8a1-4d68-a564-08d67a5343d6 admin admin] The token cache is being invalidate because project 2051ea10919f4922a6b237dbc996b6d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:46:59.915907 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-6d9503a2-d8a1-4d68-a564-08d67a5343d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:59.917351 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-6d9503a2-d8a1-4d68-a564-08d67a5343d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:46:59.920096 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1001/1997] 10.222.0.81 () {58 vars in 1232 bytes} [Wed Jul 20 22:46:59 2022] DELETE /identity/v3/projects/2051ea10919f4922a6b237dbc996b6d1 => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:46:59.924028 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98b0cb1-fec7-462f-b010-222e47e1b8dd service neutron] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:46:59.934124 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98b0cb1-fec7-462f-b010-222e47e1b8dd service neutron] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:00.011374 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3e65a2-22de-47c3-9486-827d1492a13b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:00.012372 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3e65a2-22de-47c3-9486-827d1492a13b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:00.013233 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3e65a2-22de-47c3-9486-827d1492a13b admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:00.013521 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3e65a2-22de-47c3-9486-827d1492a13b admin admin] PATH_INFO: `/v3/users/2e55d75a94e84119bc95f33f791af08d` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:00.014750 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3e65a2-22de-47c3-9486-827d1492a13b admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e55d75a94e84119bc95f33f791af08d)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:00.016732 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3e65a2-22de-47c3-9486-827d1492a13b admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:00.039309 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:47:00.375656 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-cb3e65a2-22de-47c3-9486-827d1492a13b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e55d75a94e84119bc95f33f791af08d'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:00.380213 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-cb3e65a2-22de-47c3-9486-827d1492a13b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e55d75a94e84119bc95f33f791af08d'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:00.383057 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-cb3e65a2-22de-47c3-9486-827d1492a13b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e55d75a94e84119bc95f33f791af08d'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:00.402621 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-cb3e65a2-22de-47c3-9486-827d1492a13b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e55d75a94e84119bc95f33f791af08d'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:00.467030 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-cb3e65a2-22de-47c3-9486-827d1492a13b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e55d75a94e84119bc95f33f791af08d'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:00.502950 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 997/1998] 10.222.0.81 () {58 vars in 1223 bytes} [Wed Jul 20 22:46:59 2022] DELETE /identity/v3/users/2e55d75a94e84119bc95f33f791af08d => generated 0 bytes in 579 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:47:00.820415 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7863fd8e-0e66-4755-b679-2c365c881e12 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:00.821783 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7863fd8e-0e66-4755-b679-2c365c881e12 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:00.822720 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7863fd8e-0e66-4755-b679-2c365c881e12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:00.861189 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-7863fd8e-0e66-4755-b679-2c365c881e12 None None] MFA Rules not processed for user `548054880850403ea95cf8e8b63f9b4f`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:47:00.862813 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-7863fd8e-0e66-4755-b679-2c365c881e12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:47:00.914866 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:47:01.004035 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1002/1999] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:47:00 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:47:01.020285 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3e65a2-22de-47c3-9486-827d1492a13b admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:01.021863 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-cb3e65a2-22de-47c3-9486-827d1492a13b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:47:01.104422 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3e65a2-22de-47c3-9486-827d1492a13b admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:01.109895 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8bf66f-4072-4d2f-975b-65f6a3458f05 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecf4db289fba4e4a806198c62d31f7c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:01.110795 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8bf66f-4072-4d2f-975b-65f6a3458f05 service cinder] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:01.111214 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8bf66f-4072-4d2f-975b-65f6a3458f05 service cinder] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:01.111830 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8bf66f-4072-4d2f-975b-65f6a3458f05 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:01.113855 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-3d8bf66f-4072-4d2f-975b-65f6a3458f05 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:47:01.210638 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8bf66f-4072-4d2f-975b-65f6a3458f05 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:01.215316 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8bf66f-4072-4d2f-975b-65f6a3458f05 service cinder] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:01.229216 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 998/2000] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:47:01 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:47:01.318768 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7863fd8e-0e66-4755-b679-2c365c881e12 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:01.320579 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-7863fd8e-0e66-4755-b679-2c365c881e12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:47:01.405660 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7863fd8e-0e66-4755-b679-2c365c881e12 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:01.411138 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15649765-68c8-4ea3-82e0-5fea13db547c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:01.412413 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15649765-68c8-4ea3-82e0-5fea13db547c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:01.412528 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15649765-68c8-4ea3-82e0-5fea13db547c admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:01.412789 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15649765-68c8-4ea3-82e0-5fea13db547c admin admin] PATH_INFO: `/v3/projects/9e6f824af4f34890b940123f65fe7aad` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:01.420017 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15649765-68c8-4ea3-82e0-5fea13db547c admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e6f824af4f34890b940123f65fe7aad)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:01.420017 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15649765-68c8-4ea3-82e0-5fea13db547c admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:01.600509 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-15649765-68c8-4ea3-82e0-5fea13db547c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e6f824af4f34890b940123f65fe7aad', 'request_id': 'req-15649765-68c8-4ea3-82e0-5fea13db547c'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:01.607280 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-15649765-68c8-4ea3-82e0-5fea13db547c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e6f824af4f34890b940123f65fe7aad', 'request_id': 'req-15649765-68c8-4ea3-82e0-5fea13db547c'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:01.726153 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-15649765-68c8-4ea3-82e0-5fea13db547c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e6f824af4f34890b940123f65fe7aad', 'request_id': 'req-15649765-68c8-4ea3-82e0-5fea13db547c'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:01.738593 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-15649765-68c8-4ea3-82e0-5fea13db547c admin admin] The token cache is being invalidate because project 9e6f824af4f34890b940123f65fe7aad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:47:01.738895 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-15649765-68c8-4ea3-82e0-5fea13db547c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:01.739713 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-15649765-68c8-4ea3-82e0-5fea13db547c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:01.741564 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1003/2001] 10.222.0.81 () {58 vars in 1232 bytes} [Wed Jul 20 22:47:01 2022] DELETE /identity/v3/projects/9e6f824af4f34890b940123f65fe7aad => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:47:01.746789 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dc0629-39ed-4cae-9078-a24f50d3c279 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:01.747176 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dc0629-39ed-4cae-9078-a24f50d3c279 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:01.747508 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dc0629-39ed-4cae-9078-a24f50d3c279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:01.766383 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-f4dc0629-39ed-4cae-9078-a24f50d3c279 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:47:01.767918 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-f4dc0629-39ed-4cae-9078-a24f50d3c279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:47:01.810504 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:47:01.839173 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 999/2002] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:47:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:47:01.842540 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3850a40-6c87-49a6-bf4d-74cbc0c5858a None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:01.843049 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3850a40-6c87-49a6-bf4d-74cbc0c5858a None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:01.843131 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3850a40-6c87-49a6-bf4d-74cbc0c5858a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:01.867777 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-f3850a40-6c87-49a6-bf4d-74cbc0c5858a None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:47:01.869407 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-f3850a40-6c87-49a6-bf4d-74cbc0c5858a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:47:01.881846 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1004/2003] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:47:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:47:01.883040 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dc0629-39ed-4cae-9078-a24f50d3c279 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:01.894145 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dc0629-39ed-4cae-9078-a24f50d3c279 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:01.899189 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72682960-6304-40b3-94b0-da688a246773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:01.899940 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72682960-6304-40b3-94b0-da688a246773 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:01.900304 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72682960-6304-40b3-94b0-da688a246773 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:01.900659 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72682960-6304-40b3-94b0-da688a246773 admin admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:01.902668 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72682960-6304-40b3-94b0-da688a246773 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:47:01.903146 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72682960-6304-40b3-94b0-da688a246773 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:01.905386 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72682960-6304-40b3-94b0-da688a246773 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:01.925993 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1000/2004] 10.222.0.81 () {58 vars in 1152 bytes} [Wed Jul 20 22:47:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 22:47:01.928121 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3850a40-6c87-49a6-bf4d-74cbc0c5858a None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:01.938881 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3850a40-6c87-49a6-bf4d-74cbc0c5858a None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:01.943747 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ddf435-ff70-48ed-9190-b0cbf6153f6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:01.944426 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ddf435-ff70-48ed-9190-b0cbf6153f6c admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:01.944714 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ddf435-ff70-48ed-9190-b0cbf6153f6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:01.944993 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ddf435-ff70-48ed-9190-b0cbf6153f6c admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:01.945686 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ddf435-ff70-48ed-9190-b0cbf6153f6c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:01.947920 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ddf435-ff70-48ed-9190-b0cbf6153f6c admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:02.048746 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1005/2005] 10.222.0.81 () {60 vars in 1152 bytes} [Wed Jul 20 22:47:01 2022] POST /identity/v3/projects => generated 407 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:47:02.052945 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72682960-6304-40b3-94b0-da688a246773 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:02.064778 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72682960-6304-40b3-94b0-da688a246773 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:02.122662 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0196bf00-7e26-4f94-aca9-07bfef932c5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:02.123284 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0196bf00-7e26-4f94-aca9-07bfef932c5b admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:02.123573 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0196bf00-7e26-4f94-aca9-07bfef932c5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:02.123854 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0196bf00-7e26-4f94-aca9-07bfef932c5b admin admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:02.124606 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0196bf00-7e26-4f94-aca9-07bfef932c5b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:02.126535 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0196bf00-7e26-4f94-aca9-07bfef932c5b admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:02.232260 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1001/2006] 10.222.0.81 () {60 vars in 1143 bytes} [Wed Jul 20 22:47:02 2022] POST /identity/v3/users => generated 304 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:47:02.236338 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ddf435-ff70-48ed-9190-b0cbf6153f6c admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:02.247900 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ddf435-ff70-48ed-9190-b0cbf6153f6c admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:02.254094 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430077ec-ac37-46c0-88ce-a9a1c7138bce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:02.257416 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430077ec-ac37-46c0-88ce-a9a1c7138bce admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:02.257416 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430077ec-ac37-46c0-88ce-a9a1c7138bce admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:02.257416 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430077ec-ac37-46c0-88ce-a9a1c7138bce admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:02.257416 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430077ec-ac37-46c0-88ce-a9a1c7138bce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:02.257731 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430077ec-ac37-46c0-88ce-a9a1c7138bce admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:02.275719 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1006/2007] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:47:02 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:47:02.276929 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0196bf00-7e26-4f94-aca9-07bfef932c5b admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:02.299911 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0196bf00-7e26-4f94-aca9-07bfef932c5b admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:02.305077 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856a10fe-4673-43dc-9dd4-df8aa9bbc429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:02.305759 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856a10fe-4673-43dc-9dd4-df8aa9bbc429 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:02.306050 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856a10fe-4673-43dc-9dd4-df8aa9bbc429 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:02.306331 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856a10fe-4673-43dc-9dd4-df8aa9bbc429 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:02.307231 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856a10fe-4673-43dc-9dd4-df8aa9bbc429 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:02.308831 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856a10fe-4673-43dc-9dd4-df8aa9bbc429 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:02.322899 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1002/2008] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:47:02 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:47:02.327848 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430077ec-ac37-46c0-88ce-a9a1c7138bce admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:02.339306 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430077ec-ac37-46c0-88ce-a9a1c7138bce admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:02.344282 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9f903d-0296-4441-8e53-514a5a1dab99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:02.345199 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9f903d-0296-4441-8e53-514a5a1dab99 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:02.345513 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9f903d-0296-4441-8e53-514a5a1dab99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:02.345796 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9f903d-0296-4441-8e53-514a5a1dab99 admin admin] PATH_INFO: `/v3/projects/f2b125116b98493d97cd4f0a99415afd/users/ff57fa11a3d846c08c6fcbb5d8549e14/roles/559942c4de1744169f8b5e0404ecde6b` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:02.364596 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9f903d-0296-4441-8e53-514a5a1dab99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2b125116b98493d97cd4f0a99415afd, user_id=ff57fa11a3d846c08c6fcbb5d8549e14, role_id=559942c4de1744169f8b5e0404ecde6b)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:02.364900 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9f903d-0296-4441-8e53-514a5a1dab99 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:02.368137 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:47:02.504012 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1007/2009] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:47:02 2022] PUT /identity/v3/projects/f2b125116b98493d97cd4f0a99415afd/users/ff57fa11a3d846c08c6fcbb5d8549e14/roles/559942c4de1744169f8b5e0404ecde6b => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 22:47:02.515404 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d51f9d-c092-4db1-ad71-381642210ea5 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:02.515707 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d51f9d-c092-4db1-ad71-381642210ea5 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:02.515999 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d51f9d-c092-4db1-ad71-381642210ea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:02.547997 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-89d51f9d-c092-4db1-ad71-381642210ea5 None None] MFA Rules not processed for user `ff57fa11a3d846c08c6fcbb5d8549e14`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:47:02.549267 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-89d51f9d-c092-4db1-ad71-381642210ea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:47:02.616758 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1003/2010] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:47:02 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:47:02.621348 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9f903d-0296-4441-8e53-514a5a1dab99 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:02.682383 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9f903d-0296-4441-8e53-514a5a1dab99 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:02.689178 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0aaea63-d299-4ae8-8bba-d76617546c8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:02.690009 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aaea63-d299-4ae8-8bba-d76617546c8d admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:02.690514 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aaea63-d299-4ae8-8bba-d76617546c8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:02.690988 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aaea63-d299-4ae8-8bba-d76617546c8d admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:02.691765 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0aaea63-d299-4ae8-8bba-d76617546c8d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:02.693816 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0aaea63-d299-4ae8-8bba-d76617546c8d admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:02.778911 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1008/2011] 10.222.0.81 () {60 vars in 1152 bytes} [Wed Jul 20 22:47:02 2022] POST /identity/v3/projects => generated 409 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:47:02.783245 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d51f9d-c092-4db1-ad71-381642210ea5 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:02.794717 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d51f9d-c092-4db1-ad71-381642210ea5 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:02.850106 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ad3808-185a-4f70-812c-e1158e425cf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:02.850787 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ad3808-185a-4f70-812c-e1158e425cf6 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:02.851101 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ad3808-185a-4f70-812c-e1158e425cf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:02.851384 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ad3808-185a-4f70-812c-e1158e425cf6 admin admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:02.852067 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ad3808-185a-4f70-812c-e1158e425cf6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:02.853974 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ad3808-185a-4f70-812c-e1158e425cf6 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:02.920616 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1004/2012] 10.222.0.81 () {60 vars in 1143 bytes} [Wed Jul 20 22:47:02 2022] POST /identity/v3/users => generated 311 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:47:02.924680 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0aaea63-d299-4ae8-8bba-d76617546c8d admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:02.935064 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0aaea63-d299-4ae8-8bba-d76617546c8d admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:02.940373 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618291be-e076-4326-9236-b784bc41e548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:02.941033 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618291be-e076-4326-9236-b784bc41e548 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:02.941319 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618291be-e076-4326-9236-b784bc41e548 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:02.941611 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618291be-e076-4326-9236-b784bc41e548 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:02.942335 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618291be-e076-4326-9236-b784bc41e548 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:02.943990 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618291be-e076-4326-9236-b784bc41e548 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:02.954288 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1009/2013] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:47:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:47:02.958475 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ad3808-185a-4f70-812c-e1158e425cf6 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:02.969148 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ad3808-185a-4f70-812c-e1158e425cf6 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:02.973812 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd485c4d-c7e9-496f-afc2-e890e8467454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:02.974695 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd485c4d-c7e9-496f-afc2-e890e8467454 admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:02.974982 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd485c4d-c7e9-496f-afc2-e890e8467454 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:02.975293 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd485c4d-c7e9-496f-afc2-e890e8467454 admin admin] PATH_INFO: `/v3/projects/dfac51780af04c6da9a341c1c443beca/users/343c65039a8e45a7a5079eb979f2be1c/roles/d89b6c92d35a401a8517f1d47f6be843` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:02.990343 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd485c4d-c7e9-496f-afc2-e890e8467454 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dfac51780af04c6da9a341c1c443beca, user_id=343c65039a8e45a7a5079eb979f2be1c, role_id=d89b6c92d35a401a8517f1d47f6be843)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:02.993031 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd485c4d-c7e9-496f-afc2-e890e8467454 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:03.006112 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:47:03.086090 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1005/2014] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:47:02 2022] PUT /identity/v3/projects/dfac51780af04c6da9a341c1c443beca/users/343c65039a8e45a7a5079eb979f2be1c/roles/d89b6c92d35a401a8517f1d47f6be843 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:47:03.090158 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618291be-e076-4326-9236-b784bc41e548 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:03.101220 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618291be-e076-4326-9236-b784bc41e548 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:03.163852 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271d5aed-f90f-4d58-9c95-dbea7f8a2660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:03.164570 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271d5aed-f90f-4d58-9c95-dbea7f8a2660 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:03.164908 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271d5aed-f90f-4d58-9c95-dbea7f8a2660 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:03.165215 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271d5aed-f90f-4d58-9c95-dbea7f8a2660 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:03.165948 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271d5aed-f90f-4d58-9c95-dbea7f8a2660 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:03.167579 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271d5aed-f90f-4d58-9c95-dbea7f8a2660 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:03.179108 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1010/2015] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:47:03 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 22:47:03.183847 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd485c4d-c7e9-496f-afc2-e890e8467454 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:03.194300 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd485c4d-c7e9-496f-afc2-e890e8467454 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:03.198847 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc98d810-79fd-4567-8292-7e8a9513087a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:03.199781 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc98d810-79fd-4567-8292-7e8a9513087a admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:03.200607 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc98d810-79fd-4567-8292-7e8a9513087a admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:03.200887 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc98d810-79fd-4567-8292-7e8a9513087a admin admin] PATH_INFO: `/v3/projects/dfac51780af04c6da9a341c1c443beca/users/343c65039a8e45a7a5079eb979f2be1c/roles/d89b6c92d35a401a8517f1d47f6be843` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:03.202863 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc98d810-79fd-4567-8292-7e8a9513087a admin admin] RBAC: Authorizing `identity:create_grant(project_id=dfac51780af04c6da9a341c1c443beca, user_id=343c65039a8e45a7a5079eb979f2be1c, role_id=d89b6c92d35a401a8517f1d47f6be843)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:03.205534 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc98d810-79fd-4567-8292-7e8a9513087a admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:03.216778 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1006/2016] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:47:03 2022] PUT /identity/v3/projects/dfac51780af04c6da9a341c1c443beca/users/343c65039a8e45a7a5079eb979f2be1c/roles/d89b6c92d35a401a8517f1d47f6be843 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:47:03.227037 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c272b503-3819-4861-933b-e9299db7bd9c None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:03.227341 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c272b503-3819-4861-933b-e9299db7bd9c None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:03.227625 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c272b503-3819-4861-933b-e9299db7bd9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:03.253471 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-c272b503-3819-4861-933b-e9299db7bd9c None None] MFA Rules not processed for user `343c65039a8e45a7a5079eb979f2be1c`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:47:03.254672 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-c272b503-3819-4861-933b-e9299db7bd9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:47:03.304844 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:47:03.321975 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1011/2017] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:47:03 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:47:03.333144 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc98d810-79fd-4567-8292-7e8a9513087a admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:03.334859 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-dc98d810-79fd-4567-8292-7e8a9513087a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:47:03.392649 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc98d810-79fd-4567-8292-7e8a9513087a admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:03.396470 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2c46b9-46fd-49e3-9af7-dca46bb65af7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecf4db289fba4e4a806198c62d31f7c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:03.397128 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2c46b9-46fd-49e3-9af7-dca46bb65af7 service cinder] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:03.397412 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2c46b9-46fd-49e3-9af7-dca46bb65af7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:03.397690 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2c46b9-46fd-49e3-9af7-dca46bb65af7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:03.407515 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2c46b9-46fd-49e3-9af7-dca46bb65af7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:03.408941 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2c46b9-46fd-49e3-9af7-dca46bb65af7 service cinder] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:03.418433 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1007/2018] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:47:03 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:47:04.076753 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c272b503-3819-4861-933b-e9299db7bd9c None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:04.078354 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-c272b503-3819-4861-933b-e9299db7bd9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:47:04.090208 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c272b503-3819-4861-933b-e9299db7bd9c None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:04.094464 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3787701-9892-4490-9660-90542928132c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecf4db289fba4e4a806198c62d31f7c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:04.095150 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3787701-9892-4490-9660-90542928132c service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:04.095439 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3787701-9892-4490-9660-90542928132c service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:04.095714 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3787701-9892-4490-9660-90542928132c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:04.150896 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3787701-9892-4490-9660-90542928132c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:04.152356 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3787701-9892-4490-9660-90542928132c service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:04.161889 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1012/2019] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:47:04 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:47:08.297665 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2c46b9-46fd-49e3-9af7-dca46bb65af7 service cinder] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:08.306567 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2c46b9-46fd-49e3-9af7-dca46bb65af7 service cinder] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:08.310298 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88758b84-277f-4105-bf96-d72eab4510e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecf4db289fba4e4a806198c62d31f7c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:08.310947 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88758b84-277f-4105-bf96-d72eab4510e1 service cinder] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:08.311274 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88758b84-277f-4105-bf96-d72eab4510e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:08.311552 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88758b84-277f-4105-bf96-d72eab4510e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:08.313446 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-88758b84-277f-4105-bf96-d72eab4510e1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:47:08.384997 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88758b84-277f-4105-bf96-d72eab4510e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:08.386571 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88758b84-277f-4105-bf96-d72eab4510e1 service cinder] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:08.397670 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1008/2020] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:47:08 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:47:14.157935 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=32 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:47:17.379585 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3787701-9892-4490-9660-90542928132c service cinder] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:17.381299 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-c3787701-9892-4490-9660-90542928132c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:47:17.447170 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3787701-9892-4490-9660-90542928132c service cinder] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:17.452715 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9620c83-16e8-43da-9a0d-20e94d7a9dc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:17.454755 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9620c83-16e8-43da-9a0d-20e94d7a9dc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:17.455159 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9620c83-16e8-43da-9a0d-20e94d7a9dc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:17.455450 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9620c83-16e8-43da-9a0d-20e94d7a9dc7 admin admin] PATH_INFO: `/v3/users/3f77f7db11ad43c1ae237c06dcc8baa0` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:17.456817 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9620c83-16e8-43da-9a0d-20e94d7a9dc7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f77f7db11ad43c1ae237c06dcc8baa0)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:17.458830 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9620c83-16e8-43da-9a0d-20e94d7a9dc7 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:17.694682 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-c9620c83-16e8-43da-9a0d-20e94d7a9dc7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f77f7db11ad43c1ae237c06dcc8baa0'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:17.695567 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-c9620c83-16e8-43da-9a0d-20e94d7a9dc7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f77f7db11ad43c1ae237c06dcc8baa0'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:17.696304 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-c9620c83-16e8-43da-9a0d-20e94d7a9dc7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f77f7db11ad43c1ae237c06dcc8baa0'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:17.704608 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-c9620c83-16e8-43da-9a0d-20e94d7a9dc7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f77f7db11ad43c1ae237c06dcc8baa0'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:17.788679 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-c9620c83-16e8-43da-9a0d-20e94d7a9dc7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f77f7db11ad43c1ae237c06dcc8baa0'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:17.808444 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1013/2021] 10.222.0.81 () {58 vars in 1223 bytes} [Wed Jul 20 22:47:17 2022] DELETE /identity/v3/users/3f77f7db11ad43c1ae237c06dcc8baa0 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:47:17.812923 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dfe7d2-12e9-4421-af50-30662bedaff8 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:17.813373 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dfe7d2-12e9-4421-af50-30662bedaff8 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:17.813857 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dfe7d2-12e9-4421-af50-30662bedaff8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:17.835838 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-e3dfe7d2-12e9-4421-af50-30662bedaff8 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:47:17.837068 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-e3dfe7d2-12e9-4421-af50-30662bedaff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:47:17.864120 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:47:17.900118 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1009/2022] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:47:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:47:17.910758 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9620c83-16e8-43da-9a0d-20e94d7a9dc7 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:17.912236 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-c9620c83-16e8-43da-9a0d-20e94d7a9dc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:47:17.971471 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9620c83-16e8-43da-9a0d-20e94d7a9dc7 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:17.975487 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a886afc-9ce1-484b-a397-0a66ef260886 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:17.976168 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a886afc-9ce1-484b-a397-0a66ef260886 service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:17.976456 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a886afc-9ce1-484b-a397-0a66ef260886 service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:17.976737 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a886afc-9ce1-484b-a397-0a66ef260886 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:17.987898 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a886afc-9ce1-484b-a397-0a66ef260886 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:17.989335 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a886afc-9ce1-484b-a397-0a66ef260886 service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:17.999612 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1014/2023] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:47:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:47:18.482608 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dfe7d2-12e9-4421-af50-30662bedaff8 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:18.488659 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-e3dfe7d2-12e9-4421-af50-30662bedaff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:47:18.504013 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dfe7d2-12e9-4421-af50-30662bedaff8 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:18.509613 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdde4022-32cf-4f3e-b887-e3691b2769ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:18.510594 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdde4022-32cf-4f3e-b887-e3691b2769ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:18.510780 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdde4022-32cf-4f3e-b887-e3691b2769ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:18.511082 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdde4022-32cf-4f3e-b887-e3691b2769ee admin admin] PATH_INFO: `/v3/projects/cf14585020c44f3593c109e0164639c4` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:18.517448 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdde4022-32cf-4f3e-b887-e3691b2769ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf14585020c44f3593c109e0164639c4)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:18.517448 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdde4022-32cf-4f3e-b887-e3691b2769ee admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:18.641234 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-bdde4022-32cf-4f3e-b887-e3691b2769ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf14585020c44f3593c109e0164639c4', 'request_id': 'req-bdde4022-32cf-4f3e-b887-e3691b2769ee'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:18.696491 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-bdde4022-32cf-4f3e-b887-e3691b2769ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf14585020c44f3593c109e0164639c4', 'request_id': 'req-bdde4022-32cf-4f3e-b887-e3691b2769ee'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:18.702919 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-bdde4022-32cf-4f3e-b887-e3691b2769ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf14585020c44f3593c109e0164639c4', 'request_id': 'req-bdde4022-32cf-4f3e-b887-e3691b2769ee'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:18.714713 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-bdde4022-32cf-4f3e-b887-e3691b2769ee admin admin] The token cache is being invalidate because project cf14585020c44f3593c109e0164639c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:47:18.715028 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-bdde4022-32cf-4f3e-b887-e3691b2769ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:18.716177 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-bdde4022-32cf-4f3e-b887-e3691b2769ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:18.717744 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1010/2024] 10.222.0.81 () {58 vars in 1232 bytes} [Wed Jul 20 22:47:18 2022] DELETE /identity/v3/projects/cf14585020c44f3593c109e0164639c4 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:47:18.721848 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a886afc-9ce1-484b-a397-0a66ef260886 service neutron] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:18.732337 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a886afc-9ce1-484b-a397-0a66ef260886 service neutron] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:18.751021 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:47:18.790824 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457e0986-dc33-4c06-87e2-8cfbbfc0b11e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:18.791993 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457e0986-dc33-4c06-87e2-8cfbbfc0b11e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:18.792277 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457e0986-dc33-4c06-87e2-8cfbbfc0b11e admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:18.792552 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457e0986-dc33-4c06-87e2-8cfbbfc0b11e admin admin] PATH_INFO: `/v3/users/98febf1fdc0a40f28111b5034e83ca97` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:18.795923 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457e0986-dc33-4c06-87e2-8cfbbfc0b11e admin admin] RBAC: Authorizing `identity:delete_user(user_id=98febf1fdc0a40f28111b5034e83ca97)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:18.796885 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457e0986-dc33-4c06-87e2-8cfbbfc0b11e admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:18.990706 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-457e0986-dc33-4c06-87e2-8cfbbfc0b11e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98febf1fdc0a40f28111b5034e83ca97'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:18.991323 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-457e0986-dc33-4c06-87e2-8cfbbfc0b11e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98febf1fdc0a40f28111b5034e83ca97'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:18.991984 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-457e0986-dc33-4c06-87e2-8cfbbfc0b11e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98febf1fdc0a40f28111b5034e83ca97'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:18.998796 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-457e0986-dc33-4c06-87e2-8cfbbfc0b11e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98febf1fdc0a40f28111b5034e83ca97'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:19.061684 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-457e0986-dc33-4c06-87e2-8cfbbfc0b11e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98febf1fdc0a40f28111b5034e83ca97'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:19.080973 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1015/2025] 10.222.0.81 () {58 vars in 1223 bytes} [Wed Jul 20 22:47:18 2022] DELETE /identity/v3/users/98febf1fdc0a40f28111b5034e83ca97 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:47:19.511343 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdde4022-32cf-4f3e-b887-e3691b2769ee admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:19.511343 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-bdde4022-32cf-4f3e-b887-e3691b2769ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:47:19.526829 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdde4022-32cf-4f3e-b887-e3691b2769ee admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:19.587455 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e4eb43-e8b9-47a0-b216-09a71a2a1aa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:19.588350 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e4eb43-e8b9-47a0-b216-09a71a2a1aa5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:19.588636 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e4eb43-e8b9-47a0-b216-09a71a2a1aa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:19.588927 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e4eb43-e8b9-47a0-b216-09a71a2a1aa5 admin admin] PATH_INFO: `/v3/projects/36f0533b022d49479b649b0f832eacab` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:19.590253 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e4eb43-e8b9-47a0-b216-09a71a2a1aa5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=36f0533b022d49479b649b0f832eacab)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:19.592277 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e4eb43-e8b9-47a0-b216-09a71a2a1aa5 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:19.687905 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-52e4eb43-e8b9-47a0-b216-09a71a2a1aa5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36f0533b022d49479b649b0f832eacab', 'request_id': 'req-52e4eb43-e8b9-47a0-b216-09a71a2a1aa5'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:19.691889 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:47:19.702212 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457e0986-dc33-4c06-87e2-8cfbbfc0b11e admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:19.703699 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-457e0986-dc33-4c06-87e2-8cfbbfc0b11e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:47:19.716554 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457e0986-dc33-4c06-87e2-8cfbbfc0b11e admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:19.768340 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-52e4eb43-e8b9-47a0-b216-09a71a2a1aa5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36f0533b022d49479b649b0f832eacab', 'request_id': 'req-52e4eb43-e8b9-47a0-b216-09a71a2a1aa5'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:19.775420 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-52e4eb43-e8b9-47a0-b216-09a71a2a1aa5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36f0533b022d49479b649b0f832eacab', 'request_id': 'req-52e4eb43-e8b9-47a0-b216-09a71a2a1aa5'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:19.787810 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0089fafc-91d5-4a57-bc09-c96f428e842b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:19.788765 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0089fafc-91d5-4a57-bc09-c96f428e842b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:19.789056 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0089fafc-91d5-4a57-bc09-c96f428e842b admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:19.789190 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-52e4eb43-e8b9-47a0-b216-09a71a2a1aa5 admin admin] The token cache is being invalidate because project 36f0533b022d49479b649b0f832eacab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:47:19.789327 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0089fafc-91d5-4a57-bc09-c96f428e842b admin admin] PATH_INFO: `/v3/users/ff57fa11a3d846c08c6fcbb5d8549e14` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:19.789473 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-52e4eb43-e8b9-47a0-b216-09a71a2a1aa5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:19.790004 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-52e4eb43-e8b9-47a0-b216-09a71a2a1aa5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:19.790568 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0089fafc-91d5-4a57-bc09-c96f428e842b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff57fa11a3d846c08c6fcbb5d8549e14)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:19.791634 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1011/2026] 10.222.0.81 () {58 vars in 1232 bytes} [Wed Jul 20 22:47:19 2022] DELETE /identity/v3/projects/36f0533b022d49479b649b0f832eacab => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:47:19.792691 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0089fafc-91d5-4a57-bc09-c96f428e842b admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:19.797720 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5926de-f830-481c-9af3-8abefac23ef3 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:19.798111 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5926de-f830-481c-9af3-8abefac23ef3 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:19.798534 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5926de-f830-481c-9af3-8abefac23ef3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:19.817230 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-3f5926de-f830-481c-9af3-8abefac23ef3 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:47:19.818542 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-3f5926de-f830-481c-9af3-8abefac23ef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:47:19.827129 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1012/2027] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:47:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:47:19.834745 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f07c3b-be55-4c7f-91e5-8e7d09d65275 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:19.835054 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f07c3b-be55-4c7f-91e5-8e7d09d65275 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:19.835385 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f07c3b-be55-4c7f-91e5-8e7d09d65275 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:19.852964 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-44f07c3b-be55-4c7f-91e5-8e7d09d65275 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:47:19.858358 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-44f07c3b-be55-4c7f-91e5-8e7d09d65275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:47:19.862769 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1013/2028] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:47:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 22:47:19.866752 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f07c3b-be55-4c7f-91e5-8e7d09d65275 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:19.876321 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f07c3b-be55-4c7f-91e5-8e7d09d65275 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:19.881098 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f22ad58-c701-4fc3-af2d-9ccf713b5859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:19.881842 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f22ad58-c701-4fc3-af2d-9ccf713b5859 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:19.882136 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f22ad58-c701-4fc3-af2d-9ccf713b5859 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:19.882423 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f22ad58-c701-4fc3-af2d-9ccf713b5859 admin admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:19.883177 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f22ad58-c701-4fc3-af2d-9ccf713b5859 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:47:19.883495 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f22ad58-c701-4fc3-af2d-9ccf713b5859 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:19.889964 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f22ad58-c701-4fc3-af2d-9ccf713b5859 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:19.896326 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1014/2029] 10.222.0.81 () {58 vars in 1152 bytes} [Wed Jul 20 22:47:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:47:19.900310 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f22ad58-c701-4fc3-af2d-9ccf713b5859 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:19.909865 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f22ad58-c701-4fc3-af2d-9ccf713b5859 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:19.914457 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3703f9ca-6511-4b90-a36f-caa20ced6e27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:19.915112 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3703f9ca-6511-4b90-a36f-caa20ced6e27 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:19.915404 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3703f9ca-6511-4b90-a36f-caa20ced6e27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:19.915684 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3703f9ca-6511-4b90-a36f-caa20ced6e27 admin admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:19.916359 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3703f9ca-6511-4b90-a36f-caa20ced6e27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:19.918748 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3703f9ca-6511-4b90-a36f-caa20ced6e27 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:20.007154 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1015/2030] 10.222.0.81 () {60 vars in 1152 bytes} [Wed Jul 20 22:47:19 2022] POST /identity/v3/projects => generated 369 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 22:47:20.011420 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3703f9ca-6511-4b90-a36f-caa20ced6e27 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:20.012432 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:47:20.014766 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-0089fafc-91d5-4a57-bc09-c96f428e842b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff57fa11a3d846c08c6fcbb5d8549e14'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:20.015463 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-0089fafc-91d5-4a57-bc09-c96f428e842b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff57fa11a3d846c08c6fcbb5d8549e14'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:20.016078 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-0089fafc-91d5-4a57-bc09-c96f428e842b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff57fa11a3d846c08c6fcbb5d8549e14'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:20.025392 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-0089fafc-91d5-4a57-bc09-c96f428e842b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff57fa11a3d846c08c6fcbb5d8549e14'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:20.073966 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-0089fafc-91d5-4a57-bc09-c96f428e842b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff57fa11a3d846c08c6fcbb5d8549e14'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:20.088305 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3703f9ca-6511-4b90-a36f-caa20ced6e27 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:20.094305 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d16353-3129-41b7-81a2-9fd2899b7404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:20.095004 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d16353-3129-41b7-81a2-9fd2899b7404 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:20.095004 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d16353-3129-41b7-81a2-9fd2899b7404 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:20.095617 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d16353-3129-41b7-81a2-9fd2899b7404 admin admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:20.096290 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d16353-3129-41b7-81a2-9fd2899b7404 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:20.098257 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d16353-3129-41b7-81a2-9fd2899b7404 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:20.101452 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1016/2031] 10.222.0.81 () {58 vars in 1223 bytes} [Wed Jul 20 22:47:19 2022] DELETE /identity/v3/users/ff57fa11a3d846c08c6fcbb5d8549e14 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 22:47:20.105574 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ef0c55-1964-446b-b7c2-411bf56b0d10 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:20.106420 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ef0c55-1964-446b-b7c2-411bf56b0d10 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:20.106941 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ef0c55-1964-446b-b7c2-411bf56b0d10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:20.124561 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-57ef0c55-1964-446b-b7c2-411bf56b0d10 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:47:20.125790 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-57ef0c55-1964-446b-b7c2-411bf56b0d10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:47:20.183132 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1017/2032] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:47:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 22:47:20.196086 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ef0c55-1964-446b-b7c2-411bf56b0d10 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:20.197584 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-57ef0c55-1964-446b-b7c2-411bf56b0d10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:47:20.200560 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1016/2033] 10.222.0.81 () {60 vars in 1143 bytes} [Wed Jul 20 22:47:20 2022] POST /identity/v3/users => generated 285 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:47:20.207255 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d16353-3129-41b7-81a2-9fd2899b7404 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:20.208940 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-f9d16353-3129-41b7-81a2-9fd2899b7404 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:47:20.221505 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d16353-3129-41b7-81a2-9fd2899b7404 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:20.226494 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fea7a61-26f6-4244-b439-34368eff37eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:20.227194 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fea7a61-26f6-4244-b439-34368eff37eb admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:20.227482 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fea7a61-26f6-4244-b439-34368eff37eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:20.227765 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fea7a61-26f6-4244-b439-34368eff37eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:20.228525 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fea7a61-26f6-4244-b439-34368eff37eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:20.230117 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fea7a61-26f6-4244-b439-34368eff37eb admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:20.240936 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1017/2034] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:47:20 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 22:47:20.244811 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fea7a61-26f6-4244-b439-34368eff37eb admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:20.254362 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fea7a61-26f6-4244-b439-34368eff37eb admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:20.255025 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ef0c55-1964-446b-b7c2-411bf56b0d10 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:20.258666 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b06951a-1525-4c4f-8896-17d4ff8bb450 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:20.259081 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d14faf1-9ab5-4e8c-bd43-aade4fdcbc14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:20.259349 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b06951a-1525-4c4f-8896-17d4ff8bb450 service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:20.259637 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b06951a-1525-4c4f-8896-17d4ff8bb450 service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:20.259744 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d14faf1-9ab5-4e8c-bd43-aade4fdcbc14 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:20.259918 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b06951a-1525-4c4f-8896-17d4ff8bb450 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:20.260023 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d14faf1-9ab5-4e8c-bd43-aade4fdcbc14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:20.260267 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d14faf1-9ab5-4e8c-bd43-aade4fdcbc14 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:20.260976 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d14faf1-9ab5-4e8c-bd43-aade4fdcbc14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:20.262639 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d14faf1-9ab5-4e8c-bd43-aade4fdcbc14 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:20.269014 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b06951a-1525-4c4f-8896-17d4ff8bb450 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:20.270428 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b06951a-1525-4c4f-8896-17d4ff8bb450 service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:20.273019 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1018/2035] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:47:20 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 22:47:20.277040 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d14faf1-9ab5-4e8c-bd43-aade4fdcbc14 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:20.280602 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1018/2036] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:47:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:47:20.287279 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d14faf1-9ab5-4e8c-bd43-aade4fdcbc14 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:20.292055 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9ed399-7165-4b88-8724-18be3f2c8185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:20.292932 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9ed399-7165-4b88-8724-18be3f2c8185 admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:20.293826 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9ed399-7165-4b88-8724-18be3f2c8185 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:20.294098 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9ed399-7165-4b88-8724-18be3f2c8185 admin admin] PATH_INFO: `/v3/projects/b75e3abdc8b747db9d3343c8fd554865/users/a1b7a234ddaf478f82451b5d4f629338/roles/559942c4de1744169f8b5e0404ecde6b` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:20.308820 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9ed399-7165-4b88-8724-18be3f2c8185 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b75e3abdc8b747db9d3343c8fd554865, user_id=a1b7a234ddaf478f82451b5d4f629338, role_id=559942c4de1744169f8b5e0404ecde6b)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:20.311515 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9ed399-7165-4b88-8724-18be3f2c8185 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:20.378056 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1019/2037] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:47:20 2022] PUT /identity/v3/projects/b75e3abdc8b747db9d3343c8fd554865/users/a1b7a234ddaf478f82451b5d4f629338/roles/559942c4de1744169f8b5e0404ecde6b => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 22:47:20.388952 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c08bbc2-b4c8-4725-bffb-68e4ce9f7368 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:20.389248 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c08bbc2-b4c8-4725-bffb-68e4ce9f7368 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:20.389526 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c08bbc2-b4c8-4725-bffb-68e4ce9f7368 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:20.425030 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-9c08bbc2-b4c8-4725-bffb-68e4ce9f7368 None None] MFA Rules not processed for user `a1b7a234ddaf478f82451b5d4f629338`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:47:20.426277 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-9c08bbc2-b4c8-4725-bffb-68e4ce9f7368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:47:20.489637 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:47:20.500344 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1019/2038] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:47:20 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 22:47:20.504666 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9ed399-7165-4b88-8724-18be3f2c8185 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:20.552051 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=93,keystone:DELETE=2,keystone:INSERT=5 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:47:20.573678 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9ed399-7165-4b88-8724-18be3f2c8185 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:20.578864 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce543fa-95ab-4c8b-addb-fa3e3595e539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:20.579576 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce543fa-95ab-4c8b-addb-fa3e3595e539 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:20.579857 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce543fa-95ab-4c8b-addb-fa3e3595e539 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:20.580133 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce543fa-95ab-4c8b-addb-fa3e3595e539 admin admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:20.580825 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce543fa-95ab-4c8b-addb-fa3e3595e539 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:20.582894 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce543fa-95ab-4c8b-addb-fa3e3595e539 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:20.684946 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1020/2039] 10.222.0.81 () {60 vars in 1152 bytes} [Wed Jul 20 22:47:20 2022] POST /identity/v3/projects => generated 369 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:47:20.690089 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c08bbc2-b4c8-4725-bffb-68e4ce9f7368 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:20.706531 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c08bbc2-b4c8-4725-bffb-68e4ce9f7368 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:20.786345 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db83b7c0-0ea4-4476-baa8-b3f0ae2bb122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:20.786976 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db83b7c0-0ea4-4476-baa8-b3f0ae2bb122 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:20.787292 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db83b7c0-0ea4-4476-baa8-b3f0ae2bb122 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:20.787569 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db83b7c0-0ea4-4476-baa8-b3f0ae2bb122 admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:20.788254 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db83b7c0-0ea4-4476-baa8-b3f0ae2bb122 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:20.790153 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db83b7c0-0ea4-4476-baa8-b3f0ae2bb122 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:20.829827 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce543fa-95ab-4c8b-addb-fa3e3595e539 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:20.833224 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-bce543fa-95ab-4c8b-addb-fa3e3595e539 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:47:20.847591 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce543fa-95ab-4c8b-addb-fa3e3595e539 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:20.853021 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5834a9-0bfc-4a75-99a9-aa4dad65ac3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:20.854226 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5834a9-0bfc-4a75-99a9-aa4dad65ac3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:20.854524 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5834a9-0bfc-4a75-99a9-aa4dad65ac3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:20.854826 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5834a9-0bfc-4a75-99a9-aa4dad65ac3f admin admin] PATH_INFO: `/v3/projects/f2b125116b98493d97cd4f0a99415afd` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:20.856144 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5834a9-0bfc-4a75-99a9-aa4dad65ac3f admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2b125116b98493d97cd4f0a99415afd)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:20.858053 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5834a9-0bfc-4a75-99a9-aa4dad65ac3f admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:20.898777 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1020/2040] 10.222.0.81 () {60 vars in 1143 bytes} [Wed Jul 20 22:47:20 2022] POST /identity/v3/users => generated 291 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 22:47:20.903988 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db83b7c0-0ea4-4476-baa8-b3f0ae2bb122 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:20.914294 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db83b7c0-0ea4-4476-baa8-b3f0ae2bb122 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:20.919004 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510bfe40-0d4e-47ba-b341-1e5ca6599d9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:20.919706 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510bfe40-0d4e-47ba-b341-1e5ca6599d9b admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:20.919999 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510bfe40-0d4e-47ba-b341-1e5ca6599d9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:20.920285 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510bfe40-0d4e-47ba-b341-1e5ca6599d9b admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:20.921018 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510bfe40-0d4e-47ba-b341-1e5ca6599d9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:20.922623 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510bfe40-0d4e-47ba-b341-1e5ca6599d9b admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:20.932999 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1021/2041] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:47:20 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:47:20.937451 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510bfe40-0d4e-47ba-b341-1e5ca6599d9b admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:20.946996 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510bfe40-0d4e-47ba-b341-1e5ca6599d9b admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:20.952285 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40ee4f1-3d10-4cf1-b6d6-bbd1627d1ba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:20.953164 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40ee4f1-3d10-4cf1-b6d6-bbd1627d1ba5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:20.953476 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40ee4f1-3d10-4cf1-b6d6-bbd1627d1ba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:20.953782 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40ee4f1-3d10-4cf1-b6d6-bbd1627d1ba5 admin admin] PATH_INFO: `/v3/projects/ca28bb63a01c415a993d6ae8a5a7269c/users/303a1c9f29c54641a2f22db68c1254fa/roles/d89b6c92d35a401a8517f1d47f6be843` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:20.969499 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40ee4f1-3d10-4cf1-b6d6-bbd1627d1ba5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca28bb63a01c415a993d6ae8a5a7269c, user_id=303a1c9f29c54641a2f22db68c1254fa, role_id=d89b6c92d35a401a8517f1d47f6be843)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:20.972192 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40ee4f1-3d10-4cf1-b6d6-bbd1627d1ba5 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:21.067889 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1022/2042] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:47:20 2022] PUT /identity/v3/projects/ca28bb63a01c415a993d6ae8a5a7269c/users/303a1c9f29c54641a2f22db68c1254fa/roles/d89b6c92d35a401a8517f1d47f6be843 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 22:47:21.071847 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40ee4f1-3d10-4cf1-b6d6-bbd1627d1ba5 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:21.078971 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-4f5834a9-0bfc-4a75-99a9-aa4dad65ac3f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2b125116b98493d97cd4f0a99415afd', 'request_id': 'req-4f5834a9-0bfc-4a75-99a9-aa4dad65ac3f'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:21.134907 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40ee4f1-3d10-4cf1-b6d6-bbd1627d1ba5 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:21.193672 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-4f5834a9-0bfc-4a75-99a9-aa4dad65ac3f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2b125116b98493d97cd4f0a99415afd', 'request_id': 'req-4f5834a9-0bfc-4a75-99a9-aa4dad65ac3f'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:21.195508 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:47:21.200957 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-4f5834a9-0bfc-4a75-99a9-aa4dad65ac3f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2b125116b98493d97cd4f0a99415afd', 'request_id': 'req-4f5834a9-0bfc-4a75-99a9-aa4dad65ac3f'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:21.214507 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d44b6f-f2f5-4f3f-ae1d-7371a9b27064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:21.216347 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-4f5834a9-0bfc-4a75-99a9-aa4dad65ac3f admin admin] The token cache is being invalidate because project f2b125116b98493d97cd4f0a99415afd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:47:21.216550 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d44b6f-f2f5-4f3f-ae1d-7371a9b27064 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:21.216670 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-4f5834a9-0bfc-4a75-99a9-aa4dad65ac3f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:21.216850 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d44b6f-f2f5-4f3f-ae1d-7371a9b27064 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:21.217137 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d44b6f-f2f5-4f3f-ae1d-7371a9b27064 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:21.217309 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-4f5834a9-0bfc-4a75-99a9-aa4dad65ac3f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:21.217885 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d44b6f-f2f5-4f3f-ae1d-7371a9b27064 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:21.218930 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1021/2043] 10.222.0.81 () {58 vars in 1232 bytes} [Wed Jul 20 22:47:20 2022] DELETE /identity/v3/projects/f2b125116b98493d97cd4f0a99415afd => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 22:47:21.219565 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d44b6f-f2f5-4f3f-ae1d-7371a9b27064 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:21.224799 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5834a9-0bfc-4a75-99a9-aa4dad65ac3f admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:21.226338 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-4f5834a9-0bfc-4a75-99a9-aa4dad65ac3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:47:21.232419 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1023/2044] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:47:21 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 22:47:21.242568 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d44b6f-f2f5-4f3f-ae1d-7371a9b27064 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:21.247204 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-94d44b6f-f2f5-4f3f-ae1d-7371a9b27064 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:47:21.248637 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5834a9-0bfc-4a75-99a9-aa4dad65ac3f admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:21.254427 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291df904-95f8-4732-97e5-277750fb67a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:21.255672 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291df904-95f8-4732-97e5-277750fb67a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:21.256057 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291df904-95f8-4732-97e5-277750fb67a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:21.256502 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291df904-95f8-4732-97e5-277750fb67a3 admin admin] PATH_INFO: `/v3/users/343c65039a8e45a7a5079eb979f2be1c` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:21.257947 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291df904-95f8-4732-97e5-277750fb67a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=343c65039a8e45a7a5079eb979f2be1c)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:21.260116 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291df904-95f8-4732-97e5-277750fb67a3 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:21.271326 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d44b6f-f2f5-4f3f-ae1d-7371a9b27064 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:21.276256 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6021cf2e-0137-43b9-a451-910aa3227b75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:21.277172 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6021cf2e-0137-43b9-a451-910aa3227b75 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:21.277609 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6021cf2e-0137-43b9-a451-910aa3227b75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:21.277801 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6021cf2e-0137-43b9-a451-910aa3227b75 admin admin] PATH_INFO: `/v3/projects/ca28bb63a01c415a993d6ae8a5a7269c/users/303a1c9f29c54641a2f22db68c1254fa/roles/d89b6c92d35a401a8517f1d47f6be843` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:21.289457 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6021cf2e-0137-43b9-a451-910aa3227b75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca28bb63a01c415a993d6ae8a5a7269c, user_id=303a1c9f29c54641a2f22db68c1254fa, role_id=d89b6c92d35a401a8517f1d47f6be843)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:21.289457 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6021cf2e-0137-43b9-a451-910aa3227b75 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:21.294171 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1024/2045] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:47:21 2022] PUT /identity/v3/projects/ca28bb63a01c415a993d6ae8a5a7269c/users/303a1c9f29c54641a2f22db68c1254fa/roles/d89b6c92d35a401a8517f1d47f6be843 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 22:47:21.304762 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8707ec-d236-467f-8f98-818472d24592 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:21.305051 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8707ec-d236-467f-8f98-818472d24592 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:21.305324 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8707ec-d236-467f-8f98-818472d24592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:21.441172 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-1e8707ec-d236-467f-8f98-818472d24592 None None] MFA Rules not processed for user `303a1c9f29c54641a2f22db68c1254fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:47:21.442399 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-1e8707ec-d236-467f-8f98-818472d24592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:47:21.510734 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1025/2046] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:47:21 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 22:47:21.521402 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8707ec-d236-467f-8f98-818472d24592 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:21.522735 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-1e8707ec-d236-467f-8f98-818472d24592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:47:21.567579 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-291df904-95f8-4732-97e5-277750fb67a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '343c65039a8e45a7a5079eb979f2be1c'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:21.568357 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-291df904-95f8-4732-97e5-277750fb67a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '343c65039a8e45a7a5079eb979f2be1c'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:21.569077 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-291df904-95f8-4732-97e5-277750fb67a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '343c65039a8e45a7a5079eb979f2be1c'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:21.576047 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-291df904-95f8-4732-97e5-277750fb67a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '343c65039a8e45a7a5079eb979f2be1c'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:21.579498 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8707ec-d236-467f-8f98-818472d24592 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:21.583246 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ccfb70-2c41-480a-84e0-cc0814dfbaf6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecf4db289fba4e4a806198c62d31f7c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:21.583926 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ccfb70-2c41-480a-84e0-cc0814dfbaf6 service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:21.584213 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ccfb70-2c41-480a-84e0-cc0814dfbaf6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:21.584492 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ccfb70-2c41-480a-84e0-cc0814dfbaf6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:21.586270 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-00ccfb70-2c41-480a-84e0-cc0814dfbaf6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:47:21.598202 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ccfb70-2c41-480a-84e0-cc0814dfbaf6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:21.599663 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ccfb70-2c41-480a-84e0-cc0814dfbaf6 service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:21.609499 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1026/2047] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:47:21 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 22:47:21.627404 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-291df904-95f8-4732-97e5-277750fb67a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '343c65039a8e45a7a5079eb979f2be1c'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:21.645088 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1022/2048] 10.222.0.81 () {58 vars in 1223 bytes} [Wed Jul 20 22:47:21 2022] DELETE /identity/v3/users/343c65039a8e45a7a5079eb979f2be1c => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:47:22.054018 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ccfb70-2c41-480a-84e0-cc0814dfbaf6 service cinder] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:22.055692 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-00ccfb70-2c41-480a-84e0-cc0814dfbaf6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:47:22.124255 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ccfb70-2c41-480a-84e0-cc0814dfbaf6 service cinder] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:22.134772 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:47:22.182672 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3387df64-0fee-4bd6-8940-d5f7c10c5722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:22.183600 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3387df64-0fee-4bd6-8940-d5f7c10c5722 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:22.184434 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3387df64-0fee-4bd6-8940-d5f7c10c5722 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:22.184716 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3387df64-0fee-4bd6-8940-d5f7c10c5722 admin admin] PATH_INFO: `/v3/projects/dfac51780af04c6da9a341c1c443beca` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:22.185910 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3387df64-0fee-4bd6-8940-d5f7c10c5722 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dfac51780af04c6da9a341c1c443beca)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:22.187855 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3387df64-0fee-4bd6-8940-d5f7c10c5722 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:22.304024 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-3387df64-0fee-4bd6-8940-d5f7c10c5722 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dfac51780af04c6da9a341c1c443beca', 'request_id': 'req-3387df64-0fee-4bd6-8940-d5f7c10c5722'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:22.311792 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-3387df64-0fee-4bd6-8940-d5f7c10c5722 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dfac51780af04c6da9a341c1c443beca', 'request_id': 'req-3387df64-0fee-4bd6-8940-d5f7c10c5722'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:22.507744 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-3387df64-0fee-4bd6-8940-d5f7c10c5722 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dfac51780af04c6da9a341c1c443beca', 'request_id': 'req-3387df64-0fee-4bd6-8940-d5f7c10c5722'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:22.519283 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-3387df64-0fee-4bd6-8940-d5f7c10c5722 admin admin] The token cache is being invalidate because project dfac51780af04c6da9a341c1c443beca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:47:22.519581 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-3387df64-0fee-4bd6-8940-d5f7c10c5722 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:22.520223 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-3387df64-0fee-4bd6-8940-d5f7c10c5722 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:22.521826 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1027/2049] 10.222.0.81 () {58 vars in 1232 bytes} [Wed Jul 20 22:47:22 2022] DELETE /identity/v3/projects/dfac51780af04c6da9a341c1c443beca => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:47:22.526761 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3888600c-5115-4ef4-8273-af5dded245e4 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:22.527086 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3888600c-5115-4ef4-8273-af5dded245e4 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:22.527422 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3888600c-5115-4ef4-8273-af5dded245e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:22.547750 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-3888600c-5115-4ef4-8273-af5dded245e4 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:47:22.548983 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-3888600c-5115-4ef4-8273-af5dded245e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:47:22.562816 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=80,keystone:INSERT=3,keystone:DELETE=17 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:47:22.612143 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1023/2050] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:47:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:47:22.624931 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aaa68e-3cb1-48f8-89fc-1762dd52a132 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:22.625869 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aaa68e-3cb1-48f8-89fc-1762dd52a132 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:22.625869 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aaa68e-3cb1-48f8-89fc-1762dd52a132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:22.645178 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-88aaa68e-3cb1-48f8-89fc-1762dd52a132 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:47:22.650475 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-88aaa68e-3cb1-48f8-89fc-1762dd52a132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:47:22.657987 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1028/2051] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:47:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:47:22.662100 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3888600c-5115-4ef4-8273-af5dded245e4 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:22.672763 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3888600c-5115-4ef4-8273-af5dded245e4 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:22.678138 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5b11b4-de10-4eaa-baff-49a7bff424e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:22.678819 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5b11b4-de10-4eaa-baff-49a7bff424e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:22.679139 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5b11b4-de10-4eaa-baff-49a7bff424e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:22.679424 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5b11b4-de10-4eaa-baff-49a7bff424e6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:22.680163 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5b11b4-de10-4eaa-baff-49a7bff424e6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:47:22.680479 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5b11b4-de10-4eaa-baff-49a7bff424e6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:22.682088 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5b11b4-de10-4eaa-baff-49a7bff424e6 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:22.693865 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1024/2052] 10.222.0.81 () {58 vars in 1152 bytes} [Wed Jul 20 22:47:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:47:22.698298 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aaa68e-3cb1-48f8-89fc-1762dd52a132 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:22.708311 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aaa68e-3cb1-48f8-89fc-1762dd52a132 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:22.713033 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9049dee5-08cf-486a-a23d-6164606d4eab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:22.713692 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9049dee5-08cf-486a-a23d-6164606d4eab admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:22.713977 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9049dee5-08cf-486a-a23d-6164606d4eab admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:22.714258 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9049dee5-08cf-486a-a23d-6164606d4eab admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:22.715593 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9049dee5-08cf-486a-a23d-6164606d4eab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:22.717495 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9049dee5-08cf-486a-a23d-6164606d4eab admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:22.835678 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1029/2053] 10.222.0.81 () {60 vars in 1152 bytes} [Wed Jul 20 22:47:22 2022] POST /identity/v3/projects => generated 377 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:47:22.839346 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5b11b4-de10-4eaa-baff-49a7bff424e6 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:22.850147 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5b11b4-de10-4eaa-baff-49a7bff424e6 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:22.910980 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b0617b-4aa4-4544-b623-7c2f33a859d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:22.911699 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b0617b-4aa4-4544-b623-7c2f33a859d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:22.911987 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b0617b-4aa4-4544-b623-7c2f33a859d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:22.912270 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b0617b-4aa4-4544-b623-7c2f33a859d5 admin admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:22.912976 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b0617b-4aa4-4544-b623-7c2f33a859d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:22.915652 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b0617b-4aa4-4544-b623-7c2f33a859d5 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:23.017060 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1025/2054] 10.222.0.81 () {60 vars in 1143 bytes} [Wed Jul 20 22:47:22 2022] POST /identity/v3/users => generated 289 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:47:23.024765 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9049dee5-08cf-486a-a23d-6164606d4eab admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:23.033522 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9049dee5-08cf-486a-a23d-6164606d4eab admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:23.038229 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec661d8-d7e3-4ac7-99cb-f7db539e61d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:23.038895 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec661d8-d7e3-4ac7-99cb-f7db539e61d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:23.039201 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec661d8-d7e3-4ac7-99cb-f7db539e61d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:23.039482 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec661d8-d7e3-4ac7-99cb-f7db539e61d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:23.040206 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec661d8-d7e3-4ac7-99cb-f7db539e61d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:23.041795 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec661d8-d7e3-4ac7-99cb-f7db539e61d8 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:23.052667 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1030/2055] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:47:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:47:23.058174 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b0617b-4aa4-4544-b623-7c2f33a859d5 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:23.069335 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b0617b-4aa4-4544-b623-7c2f33a859d5 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:23.074017 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776699b6-5a12-4173-ae16-e29671b95b3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:23.075403 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776699b6-5a12-4173-ae16-e29671b95b3a admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:23.075703 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776699b6-5a12-4173-ae16-e29671b95b3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:23.075969 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776699b6-5a12-4173-ae16-e29671b95b3a admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:23.076824 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776699b6-5a12-4173-ae16-e29671b95b3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:23.078424 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776699b6-5a12-4173-ae16-e29671b95b3a admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:23.092623 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1026/2056] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:47:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:47:23.101335 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec661d8-d7e3-4ac7-99cb-f7db539e61d8 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:23.103969 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec661d8-d7e3-4ac7-99cb-f7db539e61d8 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:23.108727 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639a1957-2342-4730-ab09-3dc117e72a58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:23.109628 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639a1957-2342-4730-ab09-3dc117e72a58 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:23.109918 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639a1957-2342-4730-ab09-3dc117e72a58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:23.110202 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639a1957-2342-4730-ab09-3dc117e72a58 admin admin] PATH_INFO: `/v3/projects/53569575cb2f4963a46c9d028abfc623/users/1e1a120249344c32966f8791361720e5/roles/559942c4de1744169f8b5e0404ecde6b` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:23.125443 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639a1957-2342-4730-ab09-3dc117e72a58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53569575cb2f4963a46c9d028abfc623, user_id=1e1a120249344c32966f8791361720e5, role_id=559942c4de1744169f8b5e0404ecde6b)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:23.128196 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639a1957-2342-4730-ab09-3dc117e72a58 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:23.222577 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1031/2057] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:47:23 2022] PUT /identity/v3/projects/53569575cb2f4963a46c9d028abfc623/users/1e1a120249344c32966f8791361720e5/roles/559942c4de1744169f8b5e0404ecde6b => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:47:23.234209 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f22662-de48-45d1-8514-bed71b971f17 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:23.234500 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f22662-de48-45d1-8514-bed71b971f17 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:23.234776 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f22662-de48-45d1-8514-bed71b971f17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:23.261498 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-51f22662-de48-45d1-8514-bed71b971f17 None None] MFA Rules not processed for user `1e1a120249344c32966f8791361720e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:47:23.263541 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-51f22662-de48-45d1-8514-bed71b971f17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:47:23.327546 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1027/2058] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:47:23 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:47:23.332293 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639a1957-2342-4730-ab09-3dc117e72a58 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:23.373462 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:47:23.392303 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639a1957-2342-4730-ab09-3dc117e72a58 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:23.397147 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e25bb6-f5ac-43e6-abe6-6b1b68f6de03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:23.401678 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e25bb6-f5ac-43e6-abe6-6b1b68f6de03 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:23.401678 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e25bb6-f5ac-43e6-abe6-6b1b68f6de03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:23.401678 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e25bb6-f5ac-43e6-abe6-6b1b68f6de03 admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:23.401678 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e25bb6-f5ac-43e6-abe6-6b1b68f6de03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:23.401678 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e25bb6-f5ac-43e6-abe6-6b1b68f6de03 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:23.532597 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1032/2059] 10.222.0.81 () {60 vars in 1152 bytes} [Wed Jul 20 22:47:23 2022] POST /identity/v3/projects => generated 379 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:47:23.536782 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f22662-de48-45d1-8514-bed71b971f17 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:23.548001 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f22662-de48-45d1-8514-bed71b971f17 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:23.557855 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:47:23.605214 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505da62c-ce3d-4fb0-9d14-16701e1931c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:23.606413 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505da62c-ce3d-4fb0-9d14-16701e1931c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:23.606702 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505da62c-ce3d-4fb0-9d14-16701e1931c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:23.606985 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505da62c-ce3d-4fb0-9d14-16701e1931c4 admin admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:23.607732 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505da62c-ce3d-4fb0-9d14-16701e1931c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:23.609688 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505da62c-ce3d-4fb0-9d14-16701e1931c4 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:23.759109 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1028/2060] 10.222.0.81 () {60 vars in 1143 bytes} [Wed Jul 20 22:47:23 2022] POST /identity/v3/users => generated 290 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 22:47:23.766741 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e25bb6-f5ac-43e6-abe6-6b1b68f6de03 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:23.777177 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e25bb6-f5ac-43e6-abe6-6b1b68f6de03 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:23.785625 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8aa409-ba95-43e1-871e-3499b6ed2af2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:23.785625 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8aa409-ba95-43e1-871e-3499b6ed2af2 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:23.785625 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8aa409-ba95-43e1-871e-3499b6ed2af2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:23.785625 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8aa409-ba95-43e1-871e-3499b6ed2af2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:23.788294 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8aa409-ba95-43e1-871e-3499b6ed2af2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:23.788294 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8aa409-ba95-43e1-871e-3499b6ed2af2 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:23.798338 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1033/2061] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:47:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:47:23.802800 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505da62c-ce3d-4fb0-9d14-16701e1931c4 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:23.813556 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505da62c-ce3d-4fb0-9d14-16701e1931c4 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:23.818309 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22188835-ab19-416d-aa05-52e9d1e8d783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:23.818981 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22188835-ab19-416d-aa05-52e9d1e8d783 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:23.819294 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22188835-ab19-416d-aa05-52e9d1e8d783 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:23.819572 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22188835-ab19-416d-aa05-52e9d1e8d783 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:23.820347 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22188835-ab19-416d-aa05-52e9d1e8d783 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:23.821955 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22188835-ab19-416d-aa05-52e9d1e8d783 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:23.832542 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1029/2062] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:47:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:47:23.836357 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8aa409-ba95-43e1-871e-3499b6ed2af2 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:23.847165 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8aa409-ba95-43e1-871e-3499b6ed2af2 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:23.867894 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b3a873-5f24-4372-bc6e-419c021c5e46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:23.867894 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b3a873-5f24-4372-bc6e-419c021c5e46 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:23.867894 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b3a873-5f24-4372-bc6e-419c021c5e46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:23.867894 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b3a873-5f24-4372-bc6e-419c021c5e46 admin admin] PATH_INFO: `/v3/projects/06d08e0ce30d4ccd959ab9944c53a67e/users/cd7de2ac656a44aba20f094694182d2f/roles/559942c4de1744169f8b5e0404ecde6b` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:23.871507 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b3a873-5f24-4372-bc6e-419c021c5e46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06d08e0ce30d4ccd959ab9944c53a67e, user_id=cd7de2ac656a44aba20f094694182d2f, role_id=559942c4de1744169f8b5e0404ecde6b)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:23.874330 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b3a873-5f24-4372-bc6e-419c021c5e46 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:23.998083 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1034/2063] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:47:23 2022] PUT /identity/v3/projects/06d08e0ce30d4ccd959ab9944c53a67e/users/cd7de2ac656a44aba20f094694182d2f/roles/559942c4de1744169f8b5e0404ecde6b => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 22:47:24.015938 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334ff64b-85fc-480c-af67-f8cd5d997788 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:24.015938 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334ff64b-85fc-480c-af67-f8cd5d997788 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:24.015938 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334ff64b-85fc-480c-af67-f8cd5d997788 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:24.037828 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-334ff64b-85fc-480c-af67-f8cd5d997788 None None] MFA Rules not processed for user `cd7de2ac656a44aba20f094694182d2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:47:24.039080 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-334ff64b-85fc-480c-af67-f8cd5d997788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:47:24.101301 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1030/2064] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:47:24 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:47:24.106410 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b3a873-5f24-4372-bc6e-419c021c5e46 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:24.169960 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b3a873-5f24-4372-bc6e-419c021c5e46 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:24.173735 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a0ee31-87c8-4453-9bcd-35b0dadb165a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:24.174403 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a0ee31-87c8-4453-9bcd-35b0dadb165a admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:24.174742 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a0ee31-87c8-4453-9bcd-35b0dadb165a admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:24.175034 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a0ee31-87c8-4453-9bcd-35b0dadb165a admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:24.175737 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a0ee31-87c8-4453-9bcd-35b0dadb165a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:24.177628 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a0ee31-87c8-4453-9bcd-35b0dadb165a admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:24.265687 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1035/2065] 10.222.0.81 () {60 vars in 1152 bytes} [Wed Jul 20 22:47:24 2022] POST /identity/v3/projects => generated 377 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:47:24.274144 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334ff64b-85fc-480c-af67-f8cd5d997788 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:24.286549 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334ff64b-85fc-480c-af67-f8cd5d997788 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:24.350961 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00a158a-3fbf-4068-a895-eca4a4567a42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:24.351663 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00a158a-3fbf-4068-a895-eca4a4567a42 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:24.352038 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00a158a-3fbf-4068-a895-eca4a4567a42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:24.352321 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00a158a-3fbf-4068-a895-eca4a4567a42 admin admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:24.353024 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00a158a-3fbf-4068-a895-eca4a4567a42 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:24.354945 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00a158a-3fbf-4068-a895-eca4a4567a42 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:24.478705 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1031/2066] 10.222.0.81 () {60 vars in 1143 bytes} [Wed Jul 20 22:47:24 2022] POST /identity/v3/users => generated 295 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:47:24.482149 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a0ee31-87c8-4453-9bcd-35b0dadb165a admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:24.497136 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a0ee31-87c8-4453-9bcd-35b0dadb165a admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:24.502199 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024db8c2-9cb0-43ed-8ab9-d8b29a28aa08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:24.502941 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024db8c2-9cb0-43ed-8ab9-d8b29a28aa08 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:24.503344 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024db8c2-9cb0-43ed-8ab9-d8b29a28aa08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:24.503690 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024db8c2-9cb0-43ed-8ab9-d8b29a28aa08 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:24.504736 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024db8c2-9cb0-43ed-8ab9-d8b29a28aa08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:24.507192 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024db8c2-9cb0-43ed-8ab9-d8b29a28aa08 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:24.519494 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1036/2067] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:47:24 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:47:24.527137 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00a158a-3fbf-4068-a895-eca4a4567a42 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:24.538533 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00a158a-3fbf-4068-a895-eca4a4567a42 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:24.544735 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b1c613-83be-4826-a5a0-88aed9bd77cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:24.545724 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b1c613-83be-4826-a5a0-88aed9bd77cc admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:24.546011 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b1c613-83be-4826-a5a0-88aed9bd77cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:24.546318 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b1c613-83be-4826-a5a0-88aed9bd77cc admin admin] PATH_INFO: `/v3/projects/5ad7b63751b741449dc947d5ddca07ed/users/1e6233519dd94d8b8b47e3dd66d68d31/roles/d89b6c92d35a401a8517f1d47f6be843` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:24.555752 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:47:24.562648 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b1c613-83be-4826-a5a0-88aed9bd77cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ad7b63751b741449dc947d5ddca07ed, user_id=1e6233519dd94d8b8b47e3dd66d68d31, role_id=d89b6c92d35a401a8517f1d47f6be843)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:24.565447 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b1c613-83be-4826-a5a0-88aed9bd77cc admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:24.657892 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1032/2068] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:47:24 2022] PUT /identity/v3/projects/5ad7b63751b741449dc947d5ddca07ed/users/1e6233519dd94d8b8b47e3dd66d68d31/roles/d89b6c92d35a401a8517f1d47f6be843 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:47:24.662014 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024db8c2-9cb0-43ed-8ab9-d8b29a28aa08 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:24.671955 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024db8c2-9cb0-43ed-8ab9-d8b29a28aa08 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:24.751115 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14527895-0f03-4cce-800d-774f1f0fd3ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:24.751115 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14527895-0f03-4cce-800d-774f1f0fd3ca admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:24.751115 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14527895-0f03-4cce-800d-774f1f0fd3ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:24.752080 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14527895-0f03-4cce-800d-774f1f0fd3ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:24.752615 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14527895-0f03-4cce-800d-774f1f0fd3ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:24.754248 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14527895-0f03-4cce-800d-774f1f0fd3ca admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:24.759855 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:47:24.766275 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1037/2069] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:47:24 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:47:24.770781 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b1c613-83be-4826-a5a0-88aed9bd77cc admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:24.782400 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b1c613-83be-4826-a5a0-88aed9bd77cc admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:24.787256 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49280f9d-5182-472f-a56e-289a9a6f0366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:24.788152 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49280f9d-5182-472f-a56e-289a9a6f0366 admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:24.788439 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49280f9d-5182-472f-a56e-289a9a6f0366 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:24.788720 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49280f9d-5182-472f-a56e-289a9a6f0366 admin admin] PATH_INFO: `/v3/projects/5ad7b63751b741449dc947d5ddca07ed/users/1e6233519dd94d8b8b47e3dd66d68d31/roles/d89b6c92d35a401a8517f1d47f6be843` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:24.796104 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49280f9d-5182-472f-a56e-289a9a6f0366 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ad7b63751b741449dc947d5ddca07ed, user_id=1e6233519dd94d8b8b47e3dd66d68d31, role_id=d89b6c92d35a401a8517f1d47f6be843)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:24.798799 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49280f9d-5182-472f-a56e-289a9a6f0366 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:24.809382 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1033/2070] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:47:24 2022] PUT /identity/v3/projects/5ad7b63751b741449dc947d5ddca07ed/users/1e6233519dd94d8b8b47e3dd66d68d31/roles/d89b6c92d35a401a8517f1d47f6be843 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:47:24.820868 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb15a86-162d-44ec-b5c1-6b327631db51 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:24.821166 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb15a86-162d-44ec-b5c1-6b327631db51 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:24.821454 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb15a86-162d-44ec-b5c1-6b327631db51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:24.849494 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-1fb15a86-162d-44ec-b5c1-6b327631db51 None None] MFA Rules not processed for user `1e6233519dd94d8b8b47e3dd66d68d31`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:47:24.850278 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-1fb15a86-162d-44ec-b5c1-6b327631db51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:47:24.921887 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1038/2071] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:47:24 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:47:24.933259 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49280f9d-5182-472f-a56e-289a9a6f0366 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:24.935070 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-49280f9d-5182-472f-a56e-289a9a6f0366 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:47:24.978622 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb15a86-162d-44ec-b5c1-6b327631db51 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:24.980497 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-1fb15a86-162d-44ec-b5c1-6b327631db51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:47:24.996382 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49280f9d-5182-472f-a56e-289a9a6f0366 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:25.002462 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06cc9c8-779b-4bde-92ae-a669e2f1e8dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecf4db289fba4e4a806198c62d31f7c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:25.003511 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06cc9c8-779b-4bde-92ae-a669e2f1e8dc service cinder] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:25.003985 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06cc9c8-779b-4bde-92ae-a669e2f1e8dc service cinder] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:25.008652 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06cc9c8-779b-4bde-92ae-a669e2f1e8dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:25.019673 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06cc9c8-779b-4bde-92ae-a669e2f1e8dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:25.022198 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06cc9c8-779b-4bde-92ae-a669e2f1e8dc service cinder] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:25.037427 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1034/2072] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:47:24 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:47:25.076342 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb15a86-162d-44ec-b5c1-6b327631db51 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:25.081292 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5a140b-23b2-4e7b-afed-45c1b2ce6f89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:25.082221 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5a140b-23b2-4e7b-afed-45c1b2ce6f89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:25.082508 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5a140b-23b2-4e7b-afed-45c1b2ce6f89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:25.082788 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5a140b-23b2-4e7b-afed-45c1b2ce6f89 admin admin] PATH_INFO: `/v3/users/a1b7a234ddaf478f82451b5d4f629338` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:25.084120 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5a140b-23b2-4e7b-afed-45c1b2ce6f89 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1b7a234ddaf478f82451b5d4f629338)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:25.086018 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5a140b-23b2-4e7b-afed-45c1b2ce6f89 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:25.160818 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06cc9c8-779b-4bde-92ae-a669e2f1e8dc service cinder] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:25.170102 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06cc9c8-779b-4bde-92ae-a669e2f1e8dc service cinder] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:25.173858 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d88d1af-6c02-409e-83b4-e7bb15c435c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecf4db289fba4e4a806198c62d31f7c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:25.174568 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d88d1af-6c02-409e-83b4-e7bb15c435c2 service cinder] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:25.174851 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d88d1af-6c02-409e-83b4-e7bb15c435c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:25.175160 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d88d1af-6c02-409e-83b4-e7bb15c435c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:25.234323 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d88d1af-6c02-409e-83b4-e7bb15c435c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:25.236035 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d88d1af-6c02-409e-83b4-e7bb15c435c2 service cinder] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:25.250520 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1035/2073] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:47:25 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 22:47:25.272392 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-fc5a140b-23b2-4e7b-afed-45c1b2ce6f89 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1b7a234ddaf478f82451b5d4f629338'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:25.273263 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-fc5a140b-23b2-4e7b-afed-45c1b2ce6f89 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1b7a234ddaf478f82451b5d4f629338'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:25.273912 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-fc5a140b-23b2-4e7b-afed-45c1b2ce6f89 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1b7a234ddaf478f82451b5d4f629338'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:25.283561 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-fc5a140b-23b2-4e7b-afed-45c1b2ce6f89 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1b7a234ddaf478f82451b5d4f629338'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:25.352981 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-fc5a140b-23b2-4e7b-afed-45c1b2ce6f89 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1b7a234ddaf478f82451b5d4f629338'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:25.375270 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1039/2074] 10.222.0.81 () {58 vars in 1223 bytes} [Wed Jul 20 22:47:24 2022] DELETE /identity/v3/users/a1b7a234ddaf478f82451b5d4f629338 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 22:47:25.379866 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3849c3-2ceb-424b-8ad4-07a6435d5f0a None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:25.380071 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3849c3-2ceb-424b-8ad4-07a6435d5f0a None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:25.380584 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3849c3-2ceb-424b-8ad4-07a6435d5f0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:25.400948 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-5c3849c3-2ceb-424b-8ad4-07a6435d5f0a None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:47:25.402169 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-5c3849c3-2ceb-424b-8ad4-07a6435d5f0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:47:25.474636 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1036/2075] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:47:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:47:25.485180 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5a140b-23b2-4e7b-afed-45c1b2ce6f89 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:25.489370 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-fc5a140b-23b2-4e7b-afed-45c1b2ce6f89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:47:25.496268 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:47:25.553120 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5a140b-23b2-4e7b-afed-45c1b2ce6f89 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:25.559523 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0a475a-a767-4729-873d-9188bbc4ad50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:25.560940 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0a475a-a767-4729-873d-9188bbc4ad50 service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:25.560940 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0a475a-a767-4729-873d-9188bbc4ad50 service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:25.560940 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0a475a-a767-4729-873d-9188bbc4ad50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:25.570234 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0a475a-a767-4729-873d-9188bbc4ad50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:25.571803 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0a475a-a767-4729-873d-9188bbc4ad50 service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:25.582041 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1040/2076] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:47:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:47:26.096591 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3849c3-2ceb-424b-8ad4-07a6435d5f0a None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:26.098235 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-5c3849c3-2ceb-424b-8ad4-07a6435d5f0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:47:26.118396 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3849c3-2ceb-424b-8ad4-07a6435d5f0a None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:26.137624 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f0ba28-6db1-4211-ba17-4cedd6ae2392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:26.139068 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f0ba28-6db1-4211-ba17-4cedd6ae2392 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:26.139510 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f0ba28-6db1-4211-ba17-4cedd6ae2392 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:26.139942 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f0ba28-6db1-4211-ba17-4cedd6ae2392 admin admin] PATH_INFO: `/v3/projects/b75e3abdc8b747db9d3343c8fd554865` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:26.141593 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f0ba28-6db1-4211-ba17-4cedd6ae2392 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b75e3abdc8b747db9d3343c8fd554865)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:26.143727 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f0ba28-6db1-4211-ba17-4cedd6ae2392 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:26.157108 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:47:26.300362 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-e7f0ba28-6db1-4211-ba17-4cedd6ae2392 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b75e3abdc8b747db9d3343c8fd554865', 'request_id': 'req-e7f0ba28-6db1-4211-ba17-4cedd6ae2392'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:26.376064 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-e7f0ba28-6db1-4211-ba17-4cedd6ae2392 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b75e3abdc8b747db9d3343c8fd554865', 'request_id': 'req-e7f0ba28-6db1-4211-ba17-4cedd6ae2392'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:26.383215 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-e7f0ba28-6db1-4211-ba17-4cedd6ae2392 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b75e3abdc8b747db9d3343c8fd554865', 'request_id': 'req-e7f0ba28-6db1-4211-ba17-4cedd6ae2392'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:26.394804 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-e7f0ba28-6db1-4211-ba17-4cedd6ae2392 admin admin] The token cache is being invalidate because project b75e3abdc8b747db9d3343c8fd554865 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:47:26.395132 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-e7f0ba28-6db1-4211-ba17-4cedd6ae2392 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:26.395808 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-e7f0ba28-6db1-4211-ba17-4cedd6ae2392 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:26.397581 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1037/2077] 10.222.0.81 () {58 vars in 1232 bytes} [Wed Jul 20 22:47:26 2022] DELETE /identity/v3/projects/b75e3abdc8b747db9d3343c8fd554865 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:47:26.401623 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0a475a-a767-4729-873d-9188bbc4ad50 service neutron] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:26.412748 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0a475a-a767-4729-873d-9188bbc4ad50 service neutron] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:26.470691 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72000dc6-1f9b-4d34-88b6-aa46725b0f41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:26.471882 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72000dc6-1f9b-4d34-88b6-aa46725b0f41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:26.472173 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72000dc6-1f9b-4d34-88b6-aa46725b0f41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:26.472454 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72000dc6-1f9b-4d34-88b6-aa46725b0f41 admin admin] PATH_INFO: `/v3/users/303a1c9f29c54641a2f22db68c1254fa` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:26.473692 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72000dc6-1f9b-4d34-88b6-aa46725b0f41 admin admin] RBAC: Authorizing `identity:delete_user(user_id=303a1c9f29c54641a2f22db68c1254fa)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:26.475920 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72000dc6-1f9b-4d34-88b6-aa46725b0f41 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:26.691834 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-72000dc6-1f9b-4d34-88b6-aa46725b0f41 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '303a1c9f29c54641a2f22db68c1254fa'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:26.692658 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-72000dc6-1f9b-4d34-88b6-aa46725b0f41 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '303a1c9f29c54641a2f22db68c1254fa'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:26.694170 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-72000dc6-1f9b-4d34-88b6-aa46725b0f41 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '303a1c9f29c54641a2f22db68c1254fa'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:26.703825 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-72000dc6-1f9b-4d34-88b6-aa46725b0f41 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '303a1c9f29c54641a2f22db68c1254fa'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:26.776865 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-72000dc6-1f9b-4d34-88b6-aa46725b0f41 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '303a1c9f29c54641a2f22db68c1254fa'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:26.807145 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1041/2078] 10.222.0.81 () {58 vars in 1223 bytes} [Wed Jul 20 22:47:26 2022] DELETE /identity/v3/users/303a1c9f29c54641a2f22db68c1254fa => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:47:27.340785 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f0ba28-6db1-4211-ba17-4cedd6ae2392 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:27.340785 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-e7f0ba28-6db1-4211-ba17-4cedd6ae2392 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:47:27.356390 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f0ba28-6db1-4211-ba17-4cedd6ae2392 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:27.423317 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332e4fb8-bec6-4216-9cf1-09a020e4175c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:27.423317 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332e4fb8-bec6-4216-9cf1-09a020e4175c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:27.423317 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332e4fb8-bec6-4216-9cf1-09a020e4175c admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:27.423317 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332e4fb8-bec6-4216-9cf1-09a020e4175c admin admin] PATH_INFO: `/v3/projects/ca28bb63a01c415a993d6ae8a5a7269c` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:27.424387 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332e4fb8-bec6-4216-9cf1-09a020e4175c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca28bb63a01c415a993d6ae8a5a7269c)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:27.426637 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332e4fb8-bec6-4216-9cf1-09a020e4175c admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:27.557355 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-332e4fb8-bec6-4216-9cf1-09a020e4175c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca28bb63a01c415a993d6ae8a5a7269c', 'request_id': 'req-332e4fb8-bec6-4216-9cf1-09a020e4175c'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:27.650459 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-332e4fb8-bec6-4216-9cf1-09a020e4175c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca28bb63a01c415a993d6ae8a5a7269c', 'request_id': 'req-332e4fb8-bec6-4216-9cf1-09a020e4175c'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:27.657733 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-332e4fb8-bec6-4216-9cf1-09a020e4175c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca28bb63a01c415a993d6ae8a5a7269c', 'request_id': 'req-332e4fb8-bec6-4216-9cf1-09a020e4175c'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:27.669471 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-332e4fb8-bec6-4216-9cf1-09a020e4175c admin admin] The token cache is being invalidate because project ca28bb63a01c415a993d6ae8a5a7269c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:47:27.669761 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-332e4fb8-bec6-4216-9cf1-09a020e4175c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:27.670414 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-332e4fb8-bec6-4216-9cf1-09a020e4175c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:27.672101 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1038/2079] 10.222.0.81 () {58 vars in 1232 bytes} [Wed Jul 20 22:47:27 2022] DELETE /identity/v3/projects/ca28bb63a01c415a993d6ae8a5a7269c => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:47:27.676989 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eada187a-a9d8-4b55-86d6-8a8042356f68 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:27.677478 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eada187a-a9d8-4b55-86d6-8a8042356f68 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:27.682727 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eada187a-a9d8-4b55-86d6-8a8042356f68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:27.696966 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-eada187a-a9d8-4b55-86d6-8a8042356f68 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:47:27.698214 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-eada187a-a9d8-4b55-86d6-8a8042356f68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:47:27.718999 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:47:27.780223 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1042/2080] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:47:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:47:27.788333 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba569258-bb6d-4ece-9aef-a63b5743366f None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:27.788658 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba569258-bb6d-4ece-9aef-a63b5743366f None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:27.788946 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba569258-bb6d-4ece-9aef-a63b5743366f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:27.809424 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-ba569258-bb6d-4ece-9aef-a63b5743366f None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:47:27.810807 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-ba569258-bb6d-4ece-9aef-a63b5743366f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:47:27.820104 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1039/2081] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:47:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:47:27.833359 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eada187a-a9d8-4b55-86d6-8a8042356f68 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:27.843778 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eada187a-a9d8-4b55-86d6-8a8042356f68 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:27.848599 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53747bd0-7ea8-43e8-8582-59406c27551c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:27.850965 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53747bd0-7ea8-43e8-8582-59406c27551c admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:27.851513 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53747bd0-7ea8-43e8-8582-59406c27551c admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:27.852035 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53747bd0-7ea8-43e8-8582-59406c27551c admin admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:27.853176 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53747bd0-7ea8-43e8-8582-59406c27551c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:47:27.853575 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53747bd0-7ea8-43e8-8582-59406c27551c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:27.856033 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53747bd0-7ea8-43e8-8582-59406c27551c admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:27.867674 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1043/2082] 10.222.0.81 () {58 vars in 1152 bytes} [Wed Jul 20 22:47:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:47:27.872082 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba569258-bb6d-4ece-9aef-a63b5743366f None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:27.881295 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba569258-bb6d-4ece-9aef-a63b5743366f None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:27.886172 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac76efc-3ce4-4ffa-b5f7-2fdb2e15e6f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:27.887386 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac76efc-3ce4-4ffa-b5f7-2fdb2e15e6f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:27.887672 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac76efc-3ce4-4ffa-b5f7-2fdb2e15e6f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:27.887950 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac76efc-3ce4-4ffa-b5f7-2fdb2e15e6f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:27.888700 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac76efc-3ce4-4ffa-b5f7-2fdb2e15e6f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:27.890606 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac76efc-3ce4-4ffa-b5f7-2fdb2e15e6f7 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:28.005895 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:47:28.012834 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1040/2083] 10.222.0.81 () {60 vars in 1152 bytes} [Wed Jul 20 22:47:27 2022] POST /identity/v3/projects => generated 371 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 22:47:28.017061 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53747bd0-7ea8-43e8-8582-59406c27551c admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:28.027040 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53747bd0-7ea8-43e8-8582-59406c27551c admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:28.084475 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3eef8a-3f60-4fe5-bca0-1c9fec4961c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:28.085153 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3eef8a-3f60-4fe5-bca0-1c9fec4961c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:28.085438 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3eef8a-3f60-4fe5-bca0-1c9fec4961c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:28.085717 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3eef8a-3f60-4fe5-bca0-1c9fec4961c4 admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:28.086430 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3eef8a-3f60-4fe5-bca0-1c9fec4961c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:28.088535 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3eef8a-3f60-4fe5-bca0-1c9fec4961c4 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:28.182349 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1044/2084] 10.222.0.81 () {60 vars in 1143 bytes} [Wed Jul 20 22:47:28 2022] POST /identity/v3/users => generated 286 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:47:28.186935 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac76efc-3ce4-4ffa-b5f7-2fdb2e15e6f7 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:28.197441 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac76efc-3ce4-4ffa-b5f7-2fdb2e15e6f7 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:28.202855 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a67a073-6f1c-4d17-9c01-101155db45ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:28.203590 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a67a073-6f1c-4d17-9c01-101155db45ba admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:28.203875 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a67a073-6f1c-4d17-9c01-101155db45ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:28.204153 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a67a073-6f1c-4d17-9c01-101155db45ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:28.204882 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a67a073-6f1c-4d17-9c01-101155db45ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:28.206530 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a67a073-6f1c-4d17-9c01-101155db45ba admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:28.217698 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1041/2085] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:47:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 22:47:28.221831 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3eef8a-3f60-4fe5-bca0-1c9fec4961c4 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:28.235707 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3eef8a-3f60-4fe5-bca0-1c9fec4961c4 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:28.240537 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553d058f-e95a-4391-9102-a47fbf01638c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:28.241252 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553d058f-e95a-4391-9102-a47fbf01638c admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:28.241538 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553d058f-e95a-4391-9102-a47fbf01638c admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:28.241817 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553d058f-e95a-4391-9102-a47fbf01638c admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:28.242554 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553d058f-e95a-4391-9102-a47fbf01638c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:28.244174 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553d058f-e95a-4391-9102-a47fbf01638c admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:28.255595 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1045/2086] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:47:28 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:47:28.259709 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a67a073-6f1c-4d17-9c01-101155db45ba admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:28.270918 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a67a073-6f1c-4d17-9c01-101155db45ba admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:28.276348 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ca7b7c-809f-4e3c-ae91-06bbf0db0054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:28.277766 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ca7b7c-809f-4e3c-ae91-06bbf0db0054 admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:28.278043 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ca7b7c-809f-4e3c-ae91-06bbf0db0054 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:28.278392 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ca7b7c-809f-4e3c-ae91-06bbf0db0054 admin admin] PATH_INFO: `/v3/projects/e6893b627cb04fcc9b1d61e0b599d346/users/5a44cbf059504ef8b9338a14e04d90fe/roles/559942c4de1744169f8b5e0404ecde6b` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:28.296103 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ca7b7c-809f-4e3c-ae91-06bbf0db0054 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6893b627cb04fcc9b1d61e0b599d346, user_id=5a44cbf059504ef8b9338a14e04d90fe, role_id=559942c4de1744169f8b5e0404ecde6b)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:28.298817 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ca7b7c-809f-4e3c-ae91-06bbf0db0054 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:28.376245 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1042/2087] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:47:28 2022] PUT /identity/v3/projects/e6893b627cb04fcc9b1d61e0b599d346/users/5a44cbf059504ef8b9338a14e04d90fe/roles/559942c4de1744169f8b5e0404ecde6b => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:47:28.387581 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c23051c-4ee8-48f4-a0cc-c5253929a860 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:28.387984 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c23051c-4ee8-48f4-a0cc-c5253929a860 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:28.388151 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c23051c-4ee8-48f4-a0cc-c5253929a860 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:28.415606 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-8c23051c-4ee8-48f4-a0cc-c5253929a860 None None] MFA Rules not processed for user `5a44cbf059504ef8b9338a14e04d90fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:47:28.416821 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-8c23051c-4ee8-48f4-a0cc-c5253929a860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:47:28.479124 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1046/2088] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:47:28 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:47:28.490814 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ca7b7c-809f-4e3c-ae91-06bbf0db0054 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:28.495715 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-e6ca7b7c-809f-4e3c-ae91-06bbf0db0054 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:47:28.558288 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ca7b7c-809f-4e3c-ae91-06bbf0db0054 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:28.562006 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e1ce48-20b6-4a88-8e1e-63fee6f60156 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecf4db289fba4e4a806198c62d31f7c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:28.562683 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e1ce48-20b6-4a88-8e1e-63fee6f60156 service cinder] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:28.562971 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e1ce48-20b6-4a88-8e1e-63fee6f60156 service cinder] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:28.563278 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e1ce48-20b6-4a88-8e1e-63fee6f60156 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:28.571824 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e1ce48-20b6-4a88-8e1e-63fee6f60156 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:28.573275 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e1ce48-20b6-4a88-8e1e-63fee6f60156 service cinder] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:28.582197 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1043/2089] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:47:28 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:47:28.602378 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c23051c-4ee8-48f4-a0cc-c5253929a860 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:28.628741 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:47:28.667599 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c23051c-4ee8-48f4-a0cc-c5253929a860 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:28.672208 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94adf37c-2e3c-476b-aa0b-4daf458a4e76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:28.673103 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94adf37c-2e3c-476b-aa0b-4daf458a4e76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:28.673391 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94adf37c-2e3c-476b-aa0b-4daf458a4e76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:28.673690 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94adf37c-2e3c-476b-aa0b-4daf458a4e76 admin admin] PATH_INFO: `/v3/users/5a44cbf059504ef8b9338a14e04d90fe` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:28.674915 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94adf37c-2e3c-476b-aa0b-4daf458a4e76 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a44cbf059504ef8b9338a14e04d90fe)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:28.676899 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94adf37c-2e3c-476b-aa0b-4daf458a4e76 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:28.847886 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-94adf37c-2e3c-476b-aa0b-4daf458a4e76 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a44cbf059504ef8b9338a14e04d90fe'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:28.849247 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-94adf37c-2e3c-476b-aa0b-4daf458a4e76 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a44cbf059504ef8b9338a14e04d90fe'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:28.849247 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-94adf37c-2e3c-476b-aa0b-4daf458a4e76 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a44cbf059504ef8b9338a14e04d90fe'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:28.857719 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-94adf37c-2e3c-476b-aa0b-4daf458a4e76 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a44cbf059504ef8b9338a14e04d90fe'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:28.986557 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-94adf37c-2e3c-476b-aa0b-4daf458a4e76 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a44cbf059504ef8b9338a14e04d90fe'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:28.998142 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1047/2090] 10.222.0.81 () {58 vars in 1223 bytes} [Wed Jul 20 22:47:28 2022] DELETE /identity/v3/users/5a44cbf059504ef8b9338a14e04d90fe => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:47:29.002411 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facad4cf-95b4-440e-949d-f85367687737 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:29.002775 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facad4cf-95b4-440e-949d-f85367687737 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:29.003188 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facad4cf-95b4-440e-949d-f85367687737 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:29.022044 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-facad4cf-95b4-440e-949d-f85367687737 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:47:29.023313 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-facad4cf-95b4-440e-949d-f85367687737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:47:29.084158 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1044/2091] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:47:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 22:47:29.093853 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94adf37c-2e3c-476b-aa0b-4daf458a4e76 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:29.095353 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-94adf37c-2e3c-476b-aa0b-4daf458a4e76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:47:29.156765 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94adf37c-2e3c-476b-aa0b-4daf458a4e76 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:29.160376 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec36ab48-7386-453f-a11f-d8de823b7461 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:29.161033 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec36ab48-7386-453f-a11f-d8de823b7461 service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:29.161321 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec36ab48-7386-453f-a11f-d8de823b7461 service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:29.161603 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec36ab48-7386-453f-a11f-d8de823b7461 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:29.171415 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec36ab48-7386-453f-a11f-d8de823b7461 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:29.172906 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec36ab48-7386-453f-a11f-d8de823b7461 service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:29.183217 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1048/2092] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:47:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:47:29.634255 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facad4cf-95b4-440e-949d-f85367687737 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:29.635729 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-facad4cf-95b4-440e-949d-f85367687737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:47:29.648248 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facad4cf-95b4-440e-949d-f85367687737 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:29.652928 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f212d489-6c13-43e1-a3a3-ea7eee962287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:29.653812 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f212d489-6c13-43e1-a3a3-ea7eee962287 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:29.654253 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f212d489-6c13-43e1-a3a3-ea7eee962287 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:29.654436 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f212d489-6c13-43e1-a3a3-ea7eee962287 admin admin] PATH_INFO: `/v3/projects/e6893b627cb04fcc9b1d61e0b599d346` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:29.656318 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f212d489-6c13-43e1-a3a3-ea7eee962287 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6893b627cb04fcc9b1d61e0b599d346)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:29.658256 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f212d489-6c13-43e1-a3a3-ea7eee962287 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:29.896358 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-f212d489-6c13-43e1-a3a3-ea7eee962287 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6893b627cb04fcc9b1d61e0b599d346', 'request_id': 'req-f212d489-6c13-43e1-a3a3-ea7eee962287'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:30.004392 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-f212d489-6c13-43e1-a3a3-ea7eee962287 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6893b627cb04fcc9b1d61e0b599d346', 'request_id': 'req-f212d489-6c13-43e1-a3a3-ea7eee962287'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:30.009810 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:47:30.015550 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-f212d489-6c13-43e1-a3a3-ea7eee962287 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6893b627cb04fcc9b1d61e0b599d346', 'request_id': 'req-f212d489-6c13-43e1-a3a3-ea7eee962287'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:30.022293 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-f212d489-6c13-43e1-a3a3-ea7eee962287 admin admin] The token cache is being invalidate because project e6893b627cb04fcc9b1d61e0b599d346 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:47:30.022584 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-f212d489-6c13-43e1-a3a3-ea7eee962287 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:30.023321 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-f212d489-6c13-43e1-a3a3-ea7eee962287 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:30.024897 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1045/2093] 10.222.0.81 () {58 vars in 1232 bytes} [Wed Jul 20 22:47:29 2022] DELETE /identity/v3/projects/e6893b627cb04fcc9b1d61e0b599d346 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:47:30.030327 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba01ac0-f026-4eaf-a363-a8321adec5cd None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:30.031084 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba01ac0-f026-4eaf-a363-a8321adec5cd None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:30.031370 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba01ac0-f026-4eaf-a363-a8321adec5cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:30.050455 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-8ba01ac0-f026-4eaf-a363-a8321adec5cd None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:47:30.051711 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-8ba01ac0-f026-4eaf-a363-a8321adec5cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:47:30.093446 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:47:30.112506 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1049/2094] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:47:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:47:30.119942 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9579cf32-9606-4eb6-80cc-e845cf45ded8 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:30.120236 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9579cf32-9606-4eb6-80cc-e845cf45ded8 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:30.120536 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9579cf32-9606-4eb6-80cc-e845cf45ded8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:30.139445 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-9579cf32-9606-4eb6-80cc-e845cf45ded8 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:47:30.147407 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-9579cf32-9606-4eb6-80cc-e845cf45ded8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:47:30.150033 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1046/2095] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:47:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:47:30.154810 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba01ac0-f026-4eaf-a363-a8321adec5cd None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:30.165023 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba01ac0-f026-4eaf-a363-a8321adec5cd None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:30.169796 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5896070a-2c95-4efb-8cf9-e39027f75444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:30.170410 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5896070a-2c95-4efb-8cf9-e39027f75444 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:30.170700 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5896070a-2c95-4efb-8cf9-e39027f75444 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:30.170985 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5896070a-2c95-4efb-8cf9-e39027f75444 admin admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:30.171903 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5896070a-2c95-4efb-8cf9-e39027f75444 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:47:30.172064 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5896070a-2c95-4efb-8cf9-e39027f75444 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:30.173638 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5896070a-2c95-4efb-8cf9-e39027f75444 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:30.186242 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1050/2096] 10.222.0.81 () {58 vars in 1152 bytes} [Wed Jul 20 22:47:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:47:30.190683 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9579cf32-9606-4eb6-80cc-e845cf45ded8 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:30.200840 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9579cf32-9606-4eb6-80cc-e845cf45ded8 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:30.205691 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b84e1d-d0af-479c-a87e-822d71f0cbd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:30.206458 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b84e1d-d0af-479c-a87e-822d71f0cbd2 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:30.206748 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b84e1d-d0af-479c-a87e-822d71f0cbd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:30.207053 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b84e1d-d0af-479c-a87e-822d71f0cbd2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:30.207764 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b84e1d-d0af-479c-a87e-822d71f0cbd2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:30.209664 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b84e1d-d0af-479c-a87e-822d71f0cbd2 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:30.541147 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5896070a-2c95-4efb-8cf9-e39027f75444 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:30.542947 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-5896070a-2c95-4efb-8cf9-e39027f75444 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:47:30.563552 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5896070a-2c95-4efb-8cf9-e39027f75444 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:30.563672 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1047/2097] 10.222.0.81 () {60 vars in 1152 bytes} [Wed Jul 20 22:47:30 2022] POST /identity/v3/projects => generated 373 bytes in 370 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 22:47:30.564520 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b84e1d-d0af-479c-a87e-822d71f0cbd2 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:30.626213 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fa1b4d-074b-46d7-8722-f3329653c3e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:30.627140 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fa1b4d-074b-46d7-8722-f3329653c3e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:30.627426 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fa1b4d-074b-46d7-8722-f3329653c3e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:30.627795 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fa1b4d-074b-46d7-8722-f3329653c3e6 admin admin] PATH_INFO: `/v3/users/1e1a120249344c32966f8791361720e5` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:30.629133 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fa1b4d-074b-46d7-8722-f3329653c3e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e1a120249344c32966f8791361720e5)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:30.631387 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fa1b4d-074b-46d7-8722-f3329653c3e6 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:30.635042 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b84e1d-d0af-479c-a87e-822d71f0cbd2 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:30.642407 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6681051-76f8-4380-83ad-3f0128eab0d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:30.643131 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6681051-76f8-4380-83ad-3f0128eab0d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:30.643422 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6681051-76f8-4380-83ad-3f0128eab0d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:30.643980 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6681051-76f8-4380-83ad-3f0128eab0d9 admin admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:30.644423 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6681051-76f8-4380-83ad-3f0128eab0d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:30.646339 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6681051-76f8-4380-83ad-3f0128eab0d9 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:30.800797 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1048/2098] 10.222.0.81 () {60 vars in 1143 bytes} [Wed Jul 20 22:47:30 2022] POST /identity/v3/users => generated 287 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 22:47:30.804516 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6681051-76f8-4380-83ad-3f0128eab0d9 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:30.814247 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6681051-76f8-4380-83ad-3f0128eab0d9 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:30.819022 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94516ea6-5449-4321-925c-c24714bc2555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:30.819750 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94516ea6-5449-4321-925c-c24714bc2555 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:30.820042 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94516ea6-5449-4321-925c-c24714bc2555 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:30.820328 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94516ea6-5449-4321-925c-c24714bc2555 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:30.821293 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94516ea6-5449-4321-925c-c24714bc2555 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:30.822925 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94516ea6-5449-4321-925c-c24714bc2555 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:30.836893 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1049/2099] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:47:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 22:47:30.840144 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94516ea6-5449-4321-925c-c24714bc2555 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:30.851571 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94516ea6-5449-4321-925c-c24714bc2555 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:30.860676 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d3cb2b-aeae-45d9-83d2-e15aee9752a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:30.861438 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d3cb2b-aeae-45d9-83d2-e15aee9752a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:30.861800 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d3cb2b-aeae-45d9-83d2-e15aee9752a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:30.862160 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d3cb2b-aeae-45d9-83d2-e15aee9752a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:30.863034 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d3cb2b-aeae-45d9-83d2-e15aee9752a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:30.864966 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d3cb2b-aeae-45d9-83d2-e15aee9752a1 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:30.877905 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1050/2100] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:47:30 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 22:47:30.888928 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d3cb2b-aeae-45d9-83d2-e15aee9752a1 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:30.903548 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d3cb2b-aeae-45d9-83d2-e15aee9752a1 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:30.909860 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd5f4d7-8780-4b1e-973b-8e73e0e1cc6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:30.911596 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd5f4d7-8780-4b1e-973b-8e73e0e1cc6d admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:30.911892 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd5f4d7-8780-4b1e-973b-8e73e0e1cc6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:30.912231 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd5f4d7-8780-4b1e-973b-8e73e0e1cc6d admin admin] PATH_INFO: `/v3/projects/dc95801c67d14b099cb22a22d7ce7d37/users/920686a50d04447fb36694dac11aa7af/roles/559942c4de1744169f8b5e0404ecde6b` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:30.929190 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd5f4d7-8780-4b1e-973b-8e73e0e1cc6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc95801c67d14b099cb22a22d7ce7d37, user_id=920686a50d04447fb36694dac11aa7af, role_id=559942c4de1744169f8b5e0404ecde6b)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:30.931893 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd5f4d7-8780-4b1e-973b-8e73e0e1cc6d admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:30.984538 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-49fa1b4d-074b-46d7-8722-f3329653c3e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e1a120249344c32966f8791361720e5'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:30.985207 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-49fa1b4d-074b-46d7-8722-f3329653c3e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e1a120249344c32966f8791361720e5'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:30.985822 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-49fa1b4d-074b-46d7-8722-f3329653c3e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e1a120249344c32966f8791361720e5'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:30.992595 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-49fa1b4d-074b-46d7-8722-f3329653c3e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e1a120249344c32966f8791361720e5'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:31.078995 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-49fa1b4d-074b-46d7-8722-f3329653c3e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e1a120249344c32966f8791361720e5'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:31.096376 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1051/2101] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:47:30 2022] PUT /identity/v3/projects/dc95801c67d14b099cb22a22d7ce7d37/users/920686a50d04447fb36694dac11aa7af/roles/559942c4de1744169f8b5e0404ecde6b => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 22:47:31.099058 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770082b1-6002-4981-bf98-4eb25c9ce2c2 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:31.099375 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770082b1-6002-4981-bf98-4eb25c9ce2c2 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:31.099769 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770082b1-6002-4981-bf98-4eb25c9ce2c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:31.113269 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1051/2102] 10.222.0.81 () {58 vars in 1223 bytes} [Wed Jul 20 22:47:30 2022] DELETE /identity/v3/users/1e1a120249344c32966f8791361720e5 => generated 0 bytes in 573 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:47:31.117973 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f427941e-2059-4024-9893-d4d43c534108 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:31.118267 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f427941e-2059-4024-9893-d4d43c534108 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:31.118565 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f427941e-2059-4024-9893-d4d43c534108 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:31.139128 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-f427941e-2059-4024-9893-d4d43c534108 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:47:31.140518 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-f427941e-2059-4024-9893-d4d43c534108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:47:31.140632 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-770082b1-6002-4981-bf98-4eb25c9ce2c2 None None] MFA Rules not processed for user `920686a50d04447fb36694dac11aa7af`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:47:31.141830 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-770082b1-6002-4981-bf98-4eb25c9ce2c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:47:31.220647 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1052/2103] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:47:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 22:47:31.234752 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:47:31.241821 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f427941e-2059-4024-9893-d4d43c534108 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:31.244144 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-f427941e-2059-4024-9893-d4d43c534108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:47:31.251377 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1052/2104] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:47:31 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:47:31.264076 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770082b1-6002-4981-bf98-4eb25c9ce2c2 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:31.267045 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-770082b1-6002-4981-bf98-4eb25c9ce2c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:47:31.341786 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f427941e-2059-4024-9893-d4d43c534108 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:31.341786 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5f4b14-42fe-4b16-8a5b-369d60a54cbc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:31.341786 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5f4b14-42fe-4b16-8a5b-369d60a54cbc service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:31.341786 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5f4b14-42fe-4b16-8a5b-369d60a54cbc service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:31.342134 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5f4b14-42fe-4b16-8a5b-369d60a54cbc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:31.343332 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770082b1-6002-4981-bf98-4eb25c9ce2c2 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:31.347277 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6181d16d-e396-415e-8b8c-72994af8bd32 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecf4db289fba4e4a806198c62d31f7c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:31.348001 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6181d16d-e396-415e-8b8c-72994af8bd32 service cinder] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:31.348338 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6181d16d-e396-415e-8b8c-72994af8bd32 service cinder] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:31.348625 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6181d16d-e396-415e-8b8c-72994af8bd32 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:31.348891 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5f4b14-42fe-4b16-8a5b-369d60a54cbc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:31.350528 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5f4b14-42fe-4b16-8a5b-369d60a54cbc service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:31.358387 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6181d16d-e396-415e-8b8c-72994af8bd32 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:31.360095 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6181d16d-e396-415e-8b8c-72994af8bd32 service cinder] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:31.366018 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1053/2105] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:47:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 22:47:31.369824 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1053/2106] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:47:31 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 22:47:31.570848 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:47:31.979000 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5f4b14-42fe-4b16-8a5b-369d60a54cbc service neutron] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:31.980621 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-ad5f4b14-42fe-4b16-8a5b-369d60a54cbc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:47:31.998250 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5f4b14-42fe-4b16-8a5b-369d60a54cbc service neutron] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:32.003780 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ee58c4-a342-4d39-8cfd-39ea6bbbaf7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:32.004845 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ee58c4-a342-4d39-8cfd-39ea6bbbaf7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:32.005714 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ee58c4-a342-4d39-8cfd-39ea6bbbaf7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:32.005714 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ee58c4-a342-4d39-8cfd-39ea6bbbaf7f admin admin] PATH_INFO: `/v3/projects/53569575cb2f4963a46c9d028abfc623` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:32.007431 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ee58c4-a342-4d39-8cfd-39ea6bbbaf7f admin admin] RBAC: Authorizing `identity:delete_project(project_id=53569575cb2f4963a46c9d028abfc623)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:32.016220 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ee58c4-a342-4d39-8cfd-39ea6bbbaf7f admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:32.168691 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-59ee58c4-a342-4d39-8cfd-39ea6bbbaf7f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53569575cb2f4963a46c9d028abfc623', 'request_id': 'req-59ee58c4-a342-4d39-8cfd-39ea6bbbaf7f'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:32.177184 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-59ee58c4-a342-4d39-8cfd-39ea6bbbaf7f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53569575cb2f4963a46c9d028abfc623', 'request_id': 'req-59ee58c4-a342-4d39-8cfd-39ea6bbbaf7f'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:32.302125 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-59ee58c4-a342-4d39-8cfd-39ea6bbbaf7f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53569575cb2f4963a46c9d028abfc623', 'request_id': 'req-59ee58c4-a342-4d39-8cfd-39ea6bbbaf7f'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:32.335087 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-59ee58c4-a342-4d39-8cfd-39ea6bbbaf7f admin admin] The token cache is being invalidate because project 53569575cb2f4963a46c9d028abfc623 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:47:32.335933 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-59ee58c4-a342-4d39-8cfd-39ea6bbbaf7f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:32.336707 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-59ee58c4-a342-4d39-8cfd-39ea6bbbaf7f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:32.340788 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1054/2107] 10.222.0.81 () {58 vars in 1232 bytes} [Wed Jul 20 22:47:31 2022] DELETE /identity/v3/projects/53569575cb2f4963a46c9d028abfc623 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:47:32.344896 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6181d16d-e396-415e-8b8c-72994af8bd32 service cinder] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:32.357367 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6181d16d-e396-415e-8b8c-72994af8bd32 service cinder] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:32.425218 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6d0d68-89b8-47f9-b09f-5e35069fe242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:32.426104 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6d0d68-89b8-47f9-b09f-5e35069fe242 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:32.426384 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6d0d68-89b8-47f9-b09f-5e35069fe242 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:32.426660 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6d0d68-89b8-47f9-b09f-5e35069fe242 admin admin] PATH_INFO: `/v3/users/cd7de2ac656a44aba20f094694182d2f` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:32.427876 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6d0d68-89b8-47f9-b09f-5e35069fe242 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd7de2ac656a44aba20f094694182d2f)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:32.429810 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6d0d68-89b8-47f9-b09f-5e35069fe242 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:32.867432 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-9c6d0d68-89b8-47f9-b09f-5e35069fe242 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd7de2ac656a44aba20f094694182d2f'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:32.868532 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-9c6d0d68-89b8-47f9-b09f-5e35069fe242 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd7de2ac656a44aba20f094694182d2f'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:32.869746 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-9c6d0d68-89b8-47f9-b09f-5e35069fe242 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd7de2ac656a44aba20f094694182d2f'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:32.878351 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-9c6d0d68-89b8-47f9-b09f-5e35069fe242 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd7de2ac656a44aba20f094694182d2f'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:33.052172 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-9c6d0d68-89b8-47f9-b09f-5e35069fe242 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd7de2ac656a44aba20f094694182d2f'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:33.074020 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1054/2108] 10.222.0.81 () {58 vars in 1223 bytes} [Wed Jul 20 22:47:32 2022] DELETE /identity/v3/users/cd7de2ac656a44aba20f094694182d2f => generated 0 bytes in 730 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:47:33.989292 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ee58c4-a342-4d39-8cfd-39ea6bbbaf7f admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:33.990868 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-59ee58c4-a342-4d39-8cfd-39ea6bbbaf7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:47:34.005824 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ee58c4-a342-4d39-8cfd-39ea6bbbaf7f admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:34.061681 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180bbb6f-0081-4d3c-bce5-6f83ae5f757c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:34.062563 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180bbb6f-0081-4d3c-bce5-6f83ae5f757c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:34.062846 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180bbb6f-0081-4d3c-bce5-6f83ae5f757c admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:34.063165 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180bbb6f-0081-4d3c-bce5-6f83ae5f757c admin admin] PATH_INFO: `/v3/projects/06d08e0ce30d4ccd959ab9944c53a67e` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:34.064329 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180bbb6f-0081-4d3c-bce5-6f83ae5f757c admin admin] RBAC: Authorizing `identity:delete_project(project_id=06d08e0ce30d4ccd959ab9944c53a67e)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:34.068810 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180bbb6f-0081-4d3c-bce5-6f83ae5f757c admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:34.319584 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-180bbb6f-0081-4d3c-bce5-6f83ae5f757c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06d08e0ce30d4ccd959ab9944c53a67e', 'request_id': 'req-180bbb6f-0081-4d3c-bce5-6f83ae5f757c'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:34.326184 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-180bbb6f-0081-4d3c-bce5-6f83ae5f757c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06d08e0ce30d4ccd959ab9944c53a67e', 'request_id': 'req-180bbb6f-0081-4d3c-bce5-6f83ae5f757c'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:34.667732 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-180bbb6f-0081-4d3c-bce5-6f83ae5f757c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06d08e0ce30d4ccd959ab9944c53a67e', 'request_id': 'req-180bbb6f-0081-4d3c-bce5-6f83ae5f757c'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:34.678351 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-180bbb6f-0081-4d3c-bce5-6f83ae5f757c admin admin] The token cache is being invalidate because project 06d08e0ce30d4ccd959ab9944c53a67e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:47:34.683080 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-180bbb6f-0081-4d3c-bce5-6f83ae5f757c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:34.683080 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-180bbb6f-0081-4d3c-bce5-6f83ae5f757c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:34.683080 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1055/2109] 10.222.0.81 () {58 vars in 1232 bytes} [Wed Jul 20 22:47:33 2022] DELETE /identity/v3/projects/06d08e0ce30d4ccd959ab9944c53a67e => generated 0 bytes in 692 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:47:34.685862 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6d0d68-89b8-47f9-b09f-5e35069fe242 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:34.687872 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-9c6d0d68-89b8-47f9-b09f-5e35069fe242 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:47:34.702198 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6d0d68-89b8-47f9-b09f-5e35069fe242 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:34.720787 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:47:34.771526 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5511e4-f4b8-4e59-a6bb-05374d82dda5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:34.772460 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5511e4-f4b8-4e59-a6bb-05374d82dda5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:34.772749 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5511e4-f4b8-4e59-a6bb-05374d82dda5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:34.773115 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5511e4-f4b8-4e59-a6bb-05374d82dda5 admin admin] PATH_INFO: `/v3/users/1e6233519dd94d8b8b47e3dd66d68d31` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:34.774254 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5511e4-f4b8-4e59-a6bb-05374d82dda5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e6233519dd94d8b8b47e3dd66d68d31)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:34.776289 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5511e4-f4b8-4e59-a6bb-05374d82dda5 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:35.119035 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:47:35.160793 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180bbb6f-0081-4d3c-bce5-6f83ae5f757c admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:35.162862 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-180bbb6f-0081-4d3c-bce5-6f83ae5f757c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:47:35.165099 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-3b5511e4-f4b8-4e59-a6bb-05374d82dda5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e6233519dd94d8b8b47e3dd66d68d31'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:35.165785 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-3b5511e4-f4b8-4e59-a6bb-05374d82dda5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e6233519dd94d8b8b47e3dd66d68d31'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:35.166403 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-3b5511e4-f4b8-4e59-a6bb-05374d82dda5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e6233519dd94d8b8b47e3dd66d68d31'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:35.181296 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-3b5511e4-f4b8-4e59-a6bb-05374d82dda5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e6233519dd94d8b8b47e3dd66d68d31'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:35.183858 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180bbb6f-0081-4d3c-bce5-6f83ae5f757c admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:35.188807 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2695670e-a120-406b-9e84-7b1b43648c81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:35.189959 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2695670e-a120-406b-9e84-7b1b43648c81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:35.190676 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2695670e-a120-406b-9e84-7b1b43648c81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:35.191324 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2695670e-a120-406b-9e84-7b1b43648c81 admin admin] PATH_INFO: `/v3/users/b7539f5312b84ab384fc1205518c11a6` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:35.192982 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2695670e-a120-406b-9e84-7b1b43648c81 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7539f5312b84ab384fc1205518c11a6)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:35.194925 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2695670e-a120-406b-9e84-7b1b43648c81 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:35.256500 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-3b5511e4-f4b8-4e59-a6bb-05374d82dda5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e6233519dd94d8b8b47e3dd66d68d31'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:35.284891 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1055/2110] 10.222.0.81 () {58 vars in 1223 bytes} [Wed Jul 20 22:47:34 2022] DELETE /identity/v3/users/1e6233519dd94d8b8b47e3dd66d68d31 => generated 0 bytes in 600 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 22:47:35.463261 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-2695670e-a120-406b-9e84-7b1b43648c81 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7539f5312b84ab384fc1205518c11a6'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:35.463838 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-2695670e-a120-406b-9e84-7b1b43648c81 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7539f5312b84ab384fc1205518c11a6'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:35.464662 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-2695670e-a120-406b-9e84-7b1b43648c81 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7539f5312b84ab384fc1205518c11a6'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:35.467334 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5511e4-f4b8-4e59-a6bb-05374d82dda5 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:35.468809 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-3b5511e4-f4b8-4e59-a6bb-05374d82dda5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:47:35.471750 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-2695670e-a120-406b-9e84-7b1b43648c81 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7539f5312b84ab384fc1205518c11a6'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:35.536104 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5511e4-f4b8-4e59-a6bb-05374d82dda5 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:35.551835 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6407a8ab-72fe-41f1-ae7c-a9dcc8d29942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:35.553000 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6407a8ab-72fe-41f1-ae7c-a9dcc8d29942 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:35.553452 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6407a8ab-72fe-41f1-ae7c-a9dcc8d29942 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:35.553739 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6407a8ab-72fe-41f1-ae7c-a9dcc8d29942 admin admin] PATH_INFO: `/v3/users/25255394021f4896acae48c9f2ee0c31` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:35.555285 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6407a8ab-72fe-41f1-ae7c-a9dcc8d29942 admin admin] RBAC: Authorizing `identity:delete_user(user_id=25255394021f4896acae48c9f2ee0c31)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:35.557404 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6407a8ab-72fe-41f1-ae7c-a9dcc8d29942 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:35.571285 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-2695670e-a120-406b-9e84-7b1b43648c81 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7539f5312b84ab384fc1205518c11a6'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:35.592985 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1056/2111] 10.222.0.81 () {58 vars in 1223 bytes} [Wed Jul 20 22:47:35 2022] DELETE /identity/v3/users/b7539f5312b84ab384fc1205518c11a6 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 22:47:35.830659 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2695670e-a120-406b-9e84-7b1b43648c81 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:35.832294 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-2695670e-a120-406b-9e84-7b1b43648c81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:47:35.885181 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-6407a8ab-72fe-41f1-ae7c-a9dcc8d29942 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25255394021f4896acae48c9f2ee0c31'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:35.885902 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-6407a8ab-72fe-41f1-ae7c-a9dcc8d29942 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25255394021f4896acae48c9f2ee0c31'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:35.886728 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-6407a8ab-72fe-41f1-ae7c-a9dcc8d29942 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25255394021f4896acae48c9f2ee0c31'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:35.908415 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-6407a8ab-72fe-41f1-ae7c-a9dcc8d29942 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25255394021f4896acae48c9f2ee0c31'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:35.908630 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2695670e-a120-406b-9e84-7b1b43648c81 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:35.917311 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd39430-c1fa-4a0a-8aed-1e6f32ce2878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:35.918980 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd39430-c1fa-4a0a-8aed-1e6f32ce2878 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:35.919474 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd39430-c1fa-4a0a-8aed-1e6f32ce2878 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:35.919761 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd39430-c1fa-4a0a-8aed-1e6f32ce2878 admin admin] PATH_INFO: `/v3/projects/5ad7b63751b741449dc947d5ddca07ed` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:35.921022 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd39430-c1fa-4a0a-8aed-1e6f32ce2878 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ad7b63751b741449dc947d5ddca07ed)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:35.922921 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd39430-c1fa-4a0a-8aed-1e6f32ce2878 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:36.009800 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-6407a8ab-72fe-41f1-ae7c-a9dcc8d29942 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25255394021f4896acae48c9f2ee0c31'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:36.032965 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1056/2112] 10.222.0.81 () {58 vars in 1223 bytes} [Wed Jul 20 22:47:35 2022] DELETE /identity/v3/users/25255394021f4896acae48c9f2ee0c31 => generated 0 bytes in 562 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 22:47:36.032965 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6407a8ab-72fe-41f1-ae7c-a9dcc8d29942 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:36.032965 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-6407a8ab-72fe-41f1-ae7c-a9dcc8d29942 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:47:36.077240 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-9fd39430-c1fa-4a0a-8aed-1e6f32ce2878 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ad7b63751b741449dc947d5ddca07ed', 'request_id': 'req-9fd39430-c1fa-4a0a-8aed-1e6f32ce2878'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:36.083455 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-9fd39430-c1fa-4a0a-8aed-1e6f32ce2878 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ad7b63751b741449dc947d5ddca07ed', 'request_id': 'req-9fd39430-c1fa-4a0a-8aed-1e6f32ce2878'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:36.100926 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6407a8ab-72fe-41f1-ae7c-a9dcc8d29942 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:36.172306 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b91b8b-0676-4743-af40-7ff20d2155ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:36.172306 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b91b8b-0676-4743-af40-7ff20d2155ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:36.172306 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b91b8b-0676-4743-af40-7ff20d2155ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:36.172306 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b91b8b-0676-4743-af40-7ff20d2155ca admin admin] PATH_INFO: `/v3/projects/28ff2b258273493aa3028ba5a7213eb4` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:36.172655 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b91b8b-0676-4743-af40-7ff20d2155ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=28ff2b258273493aa3028ba5a7213eb4)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:36.172655 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b91b8b-0676-4743-af40-7ff20d2155ca admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:36.193501 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-9fd39430-c1fa-4a0a-8aed-1e6f32ce2878 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ad7b63751b741449dc947d5ddca07ed', 'request_id': 'req-9fd39430-c1fa-4a0a-8aed-1e6f32ce2878'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:36.207033 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-9fd39430-c1fa-4a0a-8aed-1e6f32ce2878 admin admin] The token cache is being invalidate because project 5ad7b63751b741449dc947d5ddca07ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:47:36.207341 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-9fd39430-c1fa-4a0a-8aed-1e6f32ce2878 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:36.207993 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-9fd39430-c1fa-4a0a-8aed-1e6f32ce2878 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:36.209596 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1057/2113] 10.222.0.81 () {58 vars in 1232 bytes} [Wed Jul 20 22:47:35 2022] DELETE /identity/v3/projects/5ad7b63751b741449dc947d5ddca07ed => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 22:47:36.211388 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7de4d0f-541c-403a-a880-9bfa0de82b2f None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:36.211862 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7de4d0f-541c-403a-a880-9bfa0de82b2f None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:36.212280 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7de4d0f-541c-403a-a880-9bfa0de82b2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:36.231666 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-f7de4d0f-541c-403a-a880-9bfa0de82b2f None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:47:36.232881 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-f7de4d0f-541c-403a-a880-9bfa0de82b2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:47:36.240713 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1058/2114] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:47:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:47:36.241947 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db98f7cf-81f6-4e77-8291-1b1c6874a26a None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:36.242224 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db98f7cf-81f6-4e77-8291-1b1c6874a26a None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:36.242503 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db98f7cf-81f6-4e77-8291-1b1c6874a26a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:36.260359 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-db98f7cf-81f6-4e77-8291-1b1c6874a26a None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:47:36.262088 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-db98f7cf-81f6-4e77-8291-1b1c6874a26a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:47:36.270255 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1059/2115] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:47:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:47:36.278256 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db98f7cf-81f6-4e77-8291-1b1c6874a26a None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:36.278256 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-db98f7cf-81f6-4e77-8291-1b1c6874a26a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:47:36.288709 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-b2b91b8b-0676-4743-af40-7ff20d2155ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28ff2b258273493aa3028ba5a7213eb4', 'request_id': 'req-b2b91b8b-0676-4743-af40-7ff20d2155ca'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:36.333832 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db98f7cf-81f6-4e77-8291-1b1c6874a26a None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:36.360114 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-b2b91b8b-0676-4743-af40-7ff20d2155ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28ff2b258273493aa3028ba5a7213eb4', 'request_id': 'req-b2b91b8b-0676-4743-af40-7ff20d2155ca'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:36.368351 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-b2b91b8b-0676-4743-af40-7ff20d2155ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28ff2b258273493aa3028ba5a7213eb4', 'request_id': 'req-b2b91b8b-0676-4743-af40-7ff20d2155ca'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:36.381360 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-b2b91b8b-0676-4743-af40-7ff20d2155ca admin admin] The token cache is being invalidate because project 28ff2b258273493aa3028ba5a7213eb4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:47:36.381560 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-b2b91b8b-0676-4743-af40-7ff20d2155ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:36.382195 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-b2b91b8b-0676-4743-af40-7ff20d2155ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:36.384210 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1057/2116] 10.222.0.81 () {58 vars in 1232 bytes} [Wed Jul 20 22:47:36 2022] DELETE /identity/v3/projects/28ff2b258273493aa3028ba5a7213eb4 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:47:36.385508 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318b2d74-3f85-4d32-8160-d1f51f2b773b None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:36.385798 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318b2d74-3f85-4d32-8160-d1f51f2b773b None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:36.386592 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318b2d74-3f85-4d32-8160-d1f51f2b773b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:36.388997 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001eb9d0-6eb6-477c-ada0-8744bafeeecf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:36.389842 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001eb9d0-6eb6-477c-ada0-8744bafeeecf service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:36.390186 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001eb9d0-6eb6-477c-ada0-8744bafeeecf service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:36.390410 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001eb9d0-6eb6-477c-ada0-8744bafeeecf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:36.392810 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-001eb9d0-6eb6-477c-ada0-8744bafeeecf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:47:36.406292 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-318b2d74-3f85-4d32-8160-d1f51f2b773b None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:47:36.407555 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-318b2d74-3f85-4d32-8160-d1f51f2b773b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:47:36.468378 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001eb9d0-6eb6-477c-ada0-8744bafeeecf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:36.469862 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001eb9d0-6eb6-477c-ada0-8744bafeeecf service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:36.475937 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1058/2117] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:47:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:47:36.475937 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318b2d74-3f85-4d32-8160-d1f51f2b773b None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:36.477143 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-318b2d74-3f85-4d32-8160-d1f51f2b773b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:47:36.488498 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1060/2118] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:47:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:47:36.489412 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001eb9d0-6eb6-477c-ada0-8744bafeeecf service neutron] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:36.490334 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318b2d74-3f85-4d32-8160-d1f51f2b773b None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:36.497003 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0feb11-7c04-4d2d-b014-cba49ac4489f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:36.497914 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0feb11-7c04-4d2d-b014-cba49ac4489f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:36.498200 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0feb11-7c04-4d2d-b014-cba49ac4489f admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:36.498478 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0feb11-7c04-4d2d-b014-cba49ac4489f admin admin] PATH_INFO: `/v3/users/548054880850403ea95cf8e8b63f9b4f` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:36.499851 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0feb11-7c04-4d2d-b014-cba49ac4489f admin admin] RBAC: Authorizing `identity:delete_user(user_id=548054880850403ea95cf8e8b63f9b4f)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:36.501803 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0feb11-7c04-4d2d-b014-cba49ac4489f admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:36.502770 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001eb9d0-6eb6-477c-ada0-8744bafeeecf service neutron] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:36.507646 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381bb2ee-8bf6-4b6d-8854-5184174a6ad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:36.508504 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381bb2ee-8bf6-4b6d-8854-5184174a6ad6 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:36.508786 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381bb2ee-8bf6-4b6d-8854-5184174a6ad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:36.509233 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381bb2ee-8bf6-4b6d-8854-5184174a6ad6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:36.510011 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381bb2ee-8bf6-4b6d-8854-5184174a6ad6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:47:36.510324 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381bb2ee-8bf6-4b6d-8854-5184174a6ad6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:36.512830 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381bb2ee-8bf6-4b6d-8854-5184174a6ad6 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:36.526290 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1061/2119] 10.222.0.81 () {58 vars in 1152 bytes} [Wed Jul 20 22:47:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:47:36.530940 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381bb2ee-8bf6-4b6d-8854-5184174a6ad6 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:36.541389 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381bb2ee-8bf6-4b6d-8854-5184174a6ad6 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:36.546176 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dc8c71-8189-4269-8fde-73e774e49ef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:36.546932 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dc8c71-8189-4269-8fde-73e774e49ef9 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:36.547260 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dc8c71-8189-4269-8fde-73e774e49ef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:36.547544 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dc8c71-8189-4269-8fde-73e774e49ef9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:36.548293 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dc8c71-8189-4269-8fde-73e774e49ef9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:36.550198 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2dc8c71-8189-4269-8fde-73e774e49ef9 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:36.623325 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1062/2120] 10.222.0.81 () {60 vars in 1152 bytes} [Wed Jul 20 22:47:36 2022] POST /identity/v3/projects => generated 369 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 22:47:36.627455 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dc8c71-8189-4269-8fde-73e774e49ef9 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:36.688605 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dc8c71-8189-4269-8fde-73e774e49ef9 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:36.693879 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4538f0-f20f-40ca-886f-6ae3a6871fe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:36.694782 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4538f0-f20f-40ca-886f-6ae3a6871fe8 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:36.695251 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4538f0-f20f-40ca-886f-6ae3a6871fe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:36.695638 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4538f0-f20f-40ca-886f-6ae3a6871fe8 admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:36.696433 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4538f0-f20f-40ca-886f-6ae3a6871fe8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:36.702741 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4538f0-f20f-40ca-886f-6ae3a6871fe8 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:36.707036 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-bb0feb11-7c04-4d2d-b014-cba49ac4489f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '548054880850403ea95cf8e8b63f9b4f'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:36.707835 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-bb0feb11-7c04-4d2d-b014-cba49ac4489f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '548054880850403ea95cf8e8b63f9b4f'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:36.708644 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-bb0feb11-7c04-4d2d-b014-cba49ac4489f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '548054880850403ea95cf8e8b63f9b4f'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:36.716571 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-bb0feb11-7c04-4d2d-b014-cba49ac4489f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '548054880850403ea95cf8e8b63f9b4f'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:36.803240 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-bb0feb11-7c04-4d2d-b014-cba49ac4489f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '548054880850403ea95cf8e8b63f9b4f'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:36.812793 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1063/2121] 10.222.0.81 () {60 vars in 1143 bytes} [Wed Jul 20 22:47:36 2022] POST /identity/v3/users => generated 285 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 22:47:36.814221 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4538f0-f20f-40ca-886f-6ae3a6871fe8 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:36.816079 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-1b4538f0-f20f-40ca-886f-6ae3a6871fe8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:47:36.822130 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1059/2122] 10.222.0.81 () {58 vars in 1223 bytes} [Wed Jul 20 22:47:36 2022] DELETE /identity/v3/users/548054880850403ea95cf8e8b63f9b4f => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:47:36.830048 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4538f0-f20f-40ca-886f-6ae3a6871fe8 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:36.902486 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1840896-f353-4f86-be3c-38add25c7a2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:36.902486 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1840896-f353-4f86-be3c-38add25c7a2e admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:36.902486 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1840896-f353-4f86-be3c-38add25c7a2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:36.902486 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1840896-f353-4f86-be3c-38add25c7a2e admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:36.902787 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1840896-f353-4f86-be3c-38add25c7a2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:36.902787 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1840896-f353-4f86-be3c-38add25c7a2e admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:36.907054 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0feb11-7c04-4d2d-b014-cba49ac4489f admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:36.909053 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-bb0feb11-7c04-4d2d-b014-cba49ac4489f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:47:36.916029 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1064/2123] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:47:36 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 22:47:36.916789 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1840896-f353-4f86-be3c-38add25c7a2e admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:36.927432 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1840896-f353-4f86-be3c-38add25c7a2e admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:36.933831 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a791167-8d84-44c8-aadc-40a69f575c83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:36.934680 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a791167-8d84-44c8-aadc-40a69f575c83 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:36.935059 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a791167-8d84-44c8-aadc-40a69f575c83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:36.935361 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a791167-8d84-44c8-aadc-40a69f575c83 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:36.936091 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a791167-8d84-44c8-aadc-40a69f575c83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:36.937787 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a791167-8d84-44c8-aadc-40a69f575c83 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:36.938483 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0feb11-7c04-4d2d-b014-cba49ac4489f admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:36.945052 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560192b9-81c0-49d9-abd3-1f97d1e973e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:36.946122 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560192b9-81c0-49d9-abd3-1f97d1e973e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:36.946582 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560192b9-81c0-49d9-abd3-1f97d1e973e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:36.947050 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560192b9-81c0-49d9-abd3-1f97d1e973e3 admin admin] PATH_INFO: `/v3/projects/8e1af1488f394f1fac3ce7665da98cda` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:36.949085 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560192b9-81c0-49d9-abd3-1f97d1e973e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e1af1488f394f1fac3ce7665da98cda)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:36.949771 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1065/2124] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:47:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:47:36.952504 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560192b9-81c0-49d9-abd3-1f97d1e973e3 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:36.954028 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a791167-8d84-44c8-aadc-40a69f575c83 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:36.965914 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a791167-8d84-44c8-aadc-40a69f575c83 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:36.971238 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da229a85-f278-4b9a-9449-e24af8392c49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:36.972142 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da229a85-f278-4b9a-9449-e24af8392c49 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:36.974230 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da229a85-f278-4b9a-9449-e24af8392c49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:36.974230 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da229a85-f278-4b9a-9449-e24af8392c49 admin admin] PATH_INFO: `/v3/projects/6d6195ade06843318b16e5f12be23c38/users/661e510ad57b48a3b462577e4275474c/roles/559942c4de1744169f8b5e0404ecde6b` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:36.992997 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da229a85-f278-4b9a-9449-e24af8392c49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d6195ade06843318b16e5f12be23c38, user_id=661e510ad57b48a3b462577e4275474c, role_id=559942c4de1744169f8b5e0404ecde6b)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:36.995735 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da229a85-f278-4b9a-9449-e24af8392c49 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:37.131665 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1066/2125] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:47:36 2022] PUT /identity/v3/projects/6d6195ade06843318b16e5f12be23c38/users/661e510ad57b48a3b462577e4275474c/roles/559942c4de1744169f8b5e0404ecde6b => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:47:37.133164 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da229a85-f278-4b9a-9449-e24af8392c49 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:37.137365 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-da229a85-f278-4b9a-9449-e24af8392c49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:47:37.140982 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-560192b9-81c0-49d9-abd3-1f97d1e973e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e1af1488f394f1fac3ce7665da98cda', 'request_id': 'req-560192b9-81c0-49d9-abd3-1f97d1e973e3'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:37.209007 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da229a85-f278-4b9a-9449-e24af8392c49 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:37.213662 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3113e125-5d76-44a0-ae57-e6344e8f2b37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:37.215216 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3113e125-5d76-44a0-ae57-e6344e8f2b37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:37.215504 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3113e125-5d76-44a0-ae57-e6344e8f2b37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:37.215784 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3113e125-5d76-44a0-ae57-e6344e8f2b37 admin admin] PATH_INFO: `/v3/projects/09068fa01c6f4d3fb04164ffca5b7cee` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:37.216981 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3113e125-5d76-44a0-ae57-e6344e8f2b37 admin admin] RBAC: Authorizing `identity:delete_project(project_id=09068fa01c6f4d3fb04164ffca5b7cee)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:37.218895 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3113e125-5d76-44a0-ae57-e6344e8f2b37 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:37.255751 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-560192b9-81c0-49d9-abd3-1f97d1e973e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e1af1488f394f1fac3ce7665da98cda', 'request_id': 'req-560192b9-81c0-49d9-abd3-1f97d1e973e3'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:37.262913 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-560192b9-81c0-49d9-abd3-1f97d1e973e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e1af1488f394f1fac3ce7665da98cda', 'request_id': 'req-560192b9-81c0-49d9-abd3-1f97d1e973e3'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:37.275115 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-560192b9-81c0-49d9-abd3-1f97d1e973e3 admin admin] The token cache is being invalidate because project 8e1af1488f394f1fac3ce7665da98cda was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:47:37.275410 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-560192b9-81c0-49d9-abd3-1f97d1e973e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:37.276097 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-560192b9-81c0-49d9-abd3-1f97d1e973e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:37.277746 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1060/2126] 10.222.0.81 () {58 vars in 1232 bytes} [Wed Jul 20 22:47:36 2022] DELETE /identity/v3/projects/8e1af1488f394f1fac3ce7665da98cda => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:47:37.282977 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda76155-9216-416b-9148-a34271d7b71e None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:37.282977 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda76155-9216-416b-9148-a34271d7b71e None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:37.282977 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda76155-9216-416b-9148-a34271d7b71e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:37.306451 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-fda76155-9216-416b-9148-a34271d7b71e None None] MFA Rules not processed for user `661e510ad57b48a3b462577e4275474c`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:47:37.307712 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-fda76155-9216-416b-9148-a34271d7b71e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:47:37.374807 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1061/2127] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:47:37 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:47:37.376309 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96728b53-9c44-4ddb-b3c1-149e7cf0f353 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:37.376607 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96728b53-9c44-4ddb-b3c1-149e7cf0f353 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:37.376899 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96728b53-9c44-4ddb-b3c1-149e7cf0f353 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:37.395092 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-96728b53-9c44-4ddb-b3c1-149e7cf0f353 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:47:37.396294 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-96728b53-9c44-4ddb-b3c1-149e7cf0f353 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:47:37.404671 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1062/2128] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:47:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:47:37.407842 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96728b53-9c44-4ddb-b3c1-149e7cf0f353 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:37.409350 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-96728b53-9c44-4ddb-b3c1-149e7cf0f353 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:47:37.416461 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-3113e125-5d76-44a0-ae57-e6344e8f2b37 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09068fa01c6f4d3fb04164ffca5b7cee', 'request_id': 'req-3113e125-5d76-44a0-ae57-e6344e8f2b37'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:37.423477 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-3113e125-5d76-44a0-ae57-e6344e8f2b37 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09068fa01c6f4d3fb04164ffca5b7cee', 'request_id': 'req-3113e125-5d76-44a0-ae57-e6344e8f2b37'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:37.445125 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:47:37.470320 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96728b53-9c44-4ddb-b3c1-149e7cf0f353 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:37.501569 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-3113e125-5d76-44a0-ae57-e6344e8f2b37 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09068fa01c6f4d3fb04164ffca5b7cee', 'request_id': 'req-3113e125-5d76-44a0-ae57-e6344e8f2b37'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:37.514014 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-3113e125-5d76-44a0-ae57-e6344e8f2b37 admin admin] The token cache is being invalidate because project 09068fa01c6f4d3fb04164ffca5b7cee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:47:37.514500 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-3113e125-5d76-44a0-ae57-e6344e8f2b37 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:37.515360 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-3113e125-5d76-44a0-ae57-e6344e8f2b37 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:37.517266 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1067/2129] 10.222.0.81 () {58 vars in 1232 bytes} [Wed Jul 20 22:47:37 2022] DELETE /identity/v3/projects/09068fa01c6f4d3fb04164ffca5b7cee => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:47:37.518888 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4242b7ae-1fa5-440b-b184-7b108e79ba05 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:37.519356 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4242b7ae-1fa5-440b-b184-7b108e79ba05 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:37.520361 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4242b7ae-1fa5-440b-b184-7b108e79ba05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:37.526375 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2465f456-41d1-46b5-8227-939238256154 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c44640ea0014426a9523bb1c457778b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:37.527102 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2465f456-41d1-46b5-8227-939238256154 service glance] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:37.527417 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2465f456-41d1-46b5-8227-939238256154 service glance] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:37.527745 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2465f456-41d1-46b5-8227-939238256154 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:37.529964 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-2465f456-41d1-46b5-8227-939238256154 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:47:37.537192 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:47:37.543686 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-4242b7ae-1fa5-440b-b184-7b108e79ba05 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:47:37.544963 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-4242b7ae-1fa5-440b-b184-7b108e79ba05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:47:37.603973 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2465f456-41d1-46b5-8227-939238256154 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:37.605709 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2465f456-41d1-46b5-8227-939238256154 service glance] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:37.615352 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1063/2130] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:47:37 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:47:37.617240 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1068/2131] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:47:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:47:37.621373 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2465f456-41d1-46b5-8227-939238256154 service glance] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:37.621373 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-2465f456-41d1-46b5-8227-939238256154 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:47:37.623036 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bc31db-2045-4dcb-a2f9-7939da1c430b None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:37.623536 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bc31db-2045-4dcb-a2f9-7939da1c430b None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:37.624050 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bc31db-2045-4dcb-a2f9-7939da1c430b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:37.635099 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2465f456-41d1-46b5-8227-939238256154 service glance] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:37.641237 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d698aa-db31-408a-a1ba-fe2f7d33b2cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:37.642142 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d698aa-db31-408a-a1ba-fe2f7d33b2cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:37.642423 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d698aa-db31-408a-a1ba-fe2f7d33b2cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:37.642844 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d698aa-db31-408a-a1ba-fe2f7d33b2cd admin admin] PATH_INFO: `/v3/users/920686a50d04447fb36694dac11aa7af` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:37.644106 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d698aa-db31-408a-a1ba-fe2f7d33b2cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=920686a50d04447fb36694dac11aa7af)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:37.646036 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d698aa-db31-408a-a1ba-fe2f7d33b2cd admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:37.647147 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-b1bc31db-2045-4dcb-a2f9-7939da1c430b None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:47:37.648773 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-b1bc31db-2045-4dcb-a2f9-7939da1c430b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:47:37.659725 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1069/2132] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:47:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:47:37.660872 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bc31db-2045-4dcb-a2f9-7939da1c430b None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:37.675582 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bc31db-2045-4dcb-a2f9-7939da1c430b None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:37.680598 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe765b2b-b91e-4e33-8abe-9e65bde6d52e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:37.681383 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe765b2b-b91e-4e33-8abe-9e65bde6d52e admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:37.681813 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe765b2b-b91e-4e33-8abe-9e65bde6d52e admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:37.682231 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe765b2b-b91e-4e33-8abe-9e65bde6d52e admin admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:37.683738 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe765b2b-b91e-4e33-8abe-9e65bde6d52e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:47:37.684331 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe765b2b-b91e-4e33-8abe-9e65bde6d52e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:37.686336 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe765b2b-b91e-4e33-8abe-9e65bde6d52e admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:37.703171 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1070/2133] 10.222.0.81 () {58 vars in 1152 bytes} [Wed Jul 20 22:47:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:47:37.704926 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d258e3d8-829e-42cc-8ce4-62c3062c524f None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:37.705456 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d258e3d8-829e-42cc-8ce4-62c3062c524f None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:37.705739 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d258e3d8-829e-42cc-8ce4-62c3062c524f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:37.728599 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-d258e3d8-829e-42cc-8ce4-62c3062c524f None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:47:37.728599 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-d258e3d8-829e-42cc-8ce4-62c3062c524f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:47:37.737778 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1071/2134] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:47:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:47:37.738840 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d258e3d8-829e-42cc-8ce4-62c3062c524f None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:37.749868 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d258e3d8-829e-42cc-8ce4-62c3062c524f None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:37.755715 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb37c5f1-9647-4e5d-bef3-70c791d19f4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:37.756457 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb37c5f1-9647-4e5d-bef3-70c791d19f4f admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:37.756747 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb37c5f1-9647-4e5d-bef3-70c791d19f4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:37.757391 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb37c5f1-9647-4e5d-bef3-70c791d19f4f admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:37.757886 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb37c5f1-9647-4e5d-bef3-70c791d19f4f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:37.760052 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb37c5f1-9647-4e5d-bef3-70c791d19f4f admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:37.838000 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-25d698aa-db31-408a-a1ba-fe2f7d33b2cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '920686a50d04447fb36694dac11aa7af'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:37.838739 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-25d698aa-db31-408a-a1ba-fe2f7d33b2cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '920686a50d04447fb36694dac11aa7af'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:37.839396 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-25d698aa-db31-408a-a1ba-fe2f7d33b2cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '920686a50d04447fb36694dac11aa7af'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:37.846477 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-25d698aa-db31-408a-a1ba-fe2f7d33b2cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '920686a50d04447fb36694dac11aa7af'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:37.872077 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1072/2135] 10.222.0.81 () {60 vars in 1152 bytes} [Wed Jul 20 22:47:37 2022] POST /identity/v3/projects => generated 371 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:47:37.872077 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb37c5f1-9647-4e5d-bef3-70c791d19f4f admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:37.872077 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-cb37c5f1-9647-4e5d-bef3-70c791d19f4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:47:37.913373 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-25d698aa-db31-408a-a1ba-fe2f7d33b2cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '920686a50d04447fb36694dac11aa7af'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:37.942300 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1064/2136] 10.222.0.81 () {58 vars in 1223 bytes} [Wed Jul 20 22:47:37 2022] DELETE /identity/v3/users/920686a50d04447fb36694dac11aa7af => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:47:37.946543 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d698aa-db31-408a-a1ba-fe2f7d33b2cd admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:37.951429 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb37c5f1-9647-4e5d-bef3-70c791d19f4f admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:37.951651 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-25d698aa-db31-408a-a1ba-fe2f7d33b2cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:47:38.018261 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29201d7b-1a4c-4347-bd06-f68e2a3728b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:38.019205 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29201d7b-1a4c-4347-bd06-f68e2a3728b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:38.023916 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29201d7b-1a4c-4347-bd06-f68e2a3728b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:38.023916 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29201d7b-1a4c-4347-bd06-f68e2a3728b6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:38.023916 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29201d7b-1a4c-4347-bd06-f68e2a3728b6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:47:38.023916 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29201d7b-1a4c-4347-bd06-f68e2a3728b6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:38.023916 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29201d7b-1a4c-4347-bd06-f68e2a3728b6 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:38.027701 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d698aa-db31-408a-a1ba-fe2f7d33b2cd admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:38.032805 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed9f777-5596-4346-b719-d1560b461b43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:38.033491 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed9f777-5596-4346-b719-d1560b461b43 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:38.033801 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed9f777-5596-4346-b719-d1560b461b43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:38.034061 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed9f777-5596-4346-b719-d1560b461b43 admin admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:38.034759 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed9f777-5596-4346-b719-d1560b461b43 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:38.036751 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed9f777-5596-4346-b719-d1560b461b43 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:38.037523 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1073/2137] 10.222.0.81 () {58 vars in 1152 bytes} [Wed Jul 20 22:47:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:47:38.039842 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03313f48-1010-4111-b785-071060d4718e None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:38.040296 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03313f48-1010-4111-b785-071060d4718e None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:38.040797 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03313f48-1010-4111-b785-071060d4718e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:38.060443 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-03313f48-1010-4111-b785-071060d4718e None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:47:38.061652 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-03313f48-1010-4111-b785-071060d4718e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:47:38.069541 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1074/2138] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:47:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:47:38.070371 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03313f48-1010-4111-b785-071060d4718e None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:38.082935 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03313f48-1010-4111-b785-071060d4718e None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:38.088049 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0af9641-6cb6-4d33-aab6-de2fbb592021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:38.090493 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0af9641-6cb6-4d33-aab6-de2fbb592021 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:38.090772 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0af9641-6cb6-4d33-aab6-de2fbb592021 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:38.091086 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0af9641-6cb6-4d33-aab6-de2fbb592021 admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:38.091806 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0af9641-6cb6-4d33-aab6-de2fbb592021 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:38.094392 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0af9641-6cb6-4d33-aab6-de2fbb592021 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:38.141647 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1065/2139] 10.222.0.81 () {60 vars in 1143 bytes} [Wed Jul 20 22:47:37 2022] POST /identity/v3/users => generated 286 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:47:38.142494 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed9f777-5596-4346-b719-d1560b461b43 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:38.144231 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-aed9f777-5596-4346-b719-d1560b461b43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:47:38.211707 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed9f777-5596-4346-b719-d1560b461b43 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:38.220973 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1075/2140] 10.222.0.81 () {60 vars in 1152 bytes} [Wed Jul 20 22:47:38 2022] POST /identity/v3/projects => generated 363 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:47:38.220973 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0af9641-6cb6-4d33-aab6-de2fbb592021 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:38.224003 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73646a6-b678-4221-998b-e7d44b3961c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:38.224793 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73646a6-b678-4221-998b-e7d44b3961c7 service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:38.225162 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73646a6-b678-4221-998b-e7d44b3961c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:38.225516 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73646a6-b678-4221-998b-e7d44b3961c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:38.239335 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73646a6-b678-4221-998b-e7d44b3961c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:38.240962 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73646a6-b678-4221-998b-e7d44b3961c7 service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:38.252065 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1066/2141] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:47:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:47:38.253562 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73646a6-b678-4221-998b-e7d44b3961c7 service neutron] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:38.264363 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73646a6-b678-4221-998b-e7d44b3961c7 service neutron] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:38.315169 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0af9641-6cb6-4d33-aab6-de2fbb592021 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:38.322248 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09ede49-fde7-40d3-ba45-fde5defd3050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:38.322719 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09ede49-fde7-40d3-ba45-fde5defd3050 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:38.323027 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09ede49-fde7-40d3-ba45-fde5defd3050 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:38.323323 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09ede49-fde7-40d3-ba45-fde5defd3050 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:38.324818 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09ede49-fde7-40d3-ba45-fde5defd3050 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:38.327085 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09ede49-fde7-40d3-ba45-fde5defd3050 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:38.351772 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1076/2142] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:47:38 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:47:38.354077 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09ede49-fde7-40d3-ba45-fde5defd3050 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:38.366364 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09ede49-fde7-40d3-ba45-fde5defd3050 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:38.366788 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f493b6a8-a27c-4241-952c-445d29a96717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:38.367652 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f493b6a8-a27c-4241-952c-445d29a96717 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:38.368088 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f493b6a8-a27c-4241-952c-445d29a96717 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:38.368488 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f493b6a8-a27c-4241-952c-445d29a96717 admin admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:38.369617 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f493b6a8-a27c-4241-952c-445d29a96717 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:38.371684 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6643f49-ab7c-4a95-b60f-e6273af364a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:38.372439 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6643f49-ab7c-4a95-b60f-e6273af364a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:38.372560 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f493b6a8-a27c-4241-952c-445d29a96717 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:38.373026 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6643f49-ab7c-4a95-b60f-e6273af364a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:38.373383 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6643f49-ab7c-4a95-b60f-e6273af364a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:38.374346 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6643f49-ab7c-4a95-b60f-e6273af364a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:38.376531 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6643f49-ab7c-4a95-b60f-e6273af364a3 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:38.387195 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1077/2143] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:47:38 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:47:38.391821 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6643f49-ab7c-4a95-b60f-e6273af364a3 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:38.401238 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6643f49-ab7c-4a95-b60f-e6273af364a3 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:38.407261 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac85357f-5b7d-4ccc-9b8d-dd0bb06e8f81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:38.408335 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac85357f-5b7d-4ccc-9b8d-dd0bb06e8f81 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:38.408658 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac85357f-5b7d-4ccc-9b8d-dd0bb06e8f81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:38.408937 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac85357f-5b7d-4ccc-9b8d-dd0bb06e8f81 admin admin] PATH_INFO: `/v3/projects/5a9aff967047434097149a856bd17051/users/95d3a071e4094ffb8057b6ab505b0a62/roles/559942c4de1744169f8b5e0404ecde6b` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:38.424244 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac85357f-5b7d-4ccc-9b8d-dd0bb06e8f81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a9aff967047434097149a856bd17051, user_id=95d3a071e4094ffb8057b6ab505b0a62, role_id=559942c4de1744169f8b5e0404ecde6b)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:38.427070 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac85357f-5b7d-4ccc-9b8d-dd0bb06e8f81 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:38.539684 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1067/2144] 10.222.0.81 () {60 vars in 1143 bytes} [Wed Jul 20 22:47:38 2022] POST /identity/v3/users => generated 282 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:47:38.543636 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f493b6a8-a27c-4241-952c-445d29a96717 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:38.556110 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f493b6a8-a27c-4241-952c-445d29a96717 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:38.561813 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a7b339-7a17-45be-9cb5-3b4e47bb6f92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:38.563970 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a7b339-7a17-45be-9cb5-3b4e47bb6f92 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:38.570974 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a7b339-7a17-45be-9cb5-3b4e47bb6f92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:38.570974 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a7b339-7a17-45be-9cb5-3b4e47bb6f92 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:38.570974 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a7b339-7a17-45be-9cb5-3b4e47bb6f92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:38.574383 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1078/2145] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:47:38 2022] PUT /identity/v3/projects/5a9aff967047434097149a856bd17051/users/95d3a071e4094ffb8057b6ab505b0a62/roles/559942c4de1744169f8b5e0404ecde6b => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:47:38.578264 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a7b339-7a17-45be-9cb5-3b4e47bb6f92 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:38.590246 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556442dc-2322-4777-bf4b-ce4f139c06c8 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:38.590880 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556442dc-2322-4777-bf4b-ce4f139c06c8 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:38.591295 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556442dc-2322-4777-bf4b-ce4f139c06c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:38.600155 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1068/2146] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:47:38 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 22:47:38.606582 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a7b339-7a17-45be-9cb5-3b4e47bb6f92 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:38.626357 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a7b339-7a17-45be-9cb5-3b4e47bb6f92 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:38.642791 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-556442dc-2322-4777-bf4b-ce4f139c06c8 None None] MFA Rules not processed for user `95d3a071e4094ffb8057b6ab505b0a62`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:47:38.645949 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-556442dc-2322-4777-bf4b-ce4f139c06c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:47:38.715582 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c101b065-d268-4e57-8fe8-9d689bc46d00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:38.716551 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c101b065-d268-4e57-8fe8-9d689bc46d00 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:38.716982 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c101b065-d268-4e57-8fe8-9d689bc46d00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:38.717407 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c101b065-d268-4e57-8fe8-9d689bc46d00 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:38.718326 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c101b065-d268-4e57-8fe8-9d689bc46d00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:38.720753 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c101b065-d268-4e57-8fe8-9d689bc46d00 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:38.738417 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1069/2147] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:47:38 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 22:47:38.751838 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c101b065-d268-4e57-8fe8-9d689bc46d00 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:38.763941 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c101b065-d268-4e57-8fe8-9d689bc46d00 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:38.770020 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e28d04-03b3-492e-a127-b4daedfda1ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:38.771241 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e28d04-03b3-492e-a127-b4daedfda1ea admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:38.771748 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e28d04-03b3-492e-a127-b4daedfda1ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:38.772129 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e28d04-03b3-492e-a127-b4daedfda1ea admin admin] PATH_INFO: `/v3/projects/23671aa0492d49eb9749e5dc12643038/users/de57e5e148804737bf51412acc4d7cc0/roles/559942c4de1744169f8b5e0404ecde6b` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:38.782706 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1079/2148] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:47:38 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:47:38.791313 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e28d04-03b3-492e-a127-b4daedfda1ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=23671aa0492d49eb9749e5dc12643038, user_id=de57e5e148804737bf51412acc4d7cc0, role_id=559942c4de1744169f8b5e0404ecde6b)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:38.794411 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e28d04-03b3-492e-a127-b4daedfda1ea admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:38.795535 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556442dc-2322-4777-bf4b-ce4f139c06c8 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:38.797708 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-556442dc-2322-4777-bf4b-ce4f139c06c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:47:38.858616 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556442dc-2322-4777-bf4b-ce4f139c06c8 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:38.873334 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd5c5d8-a6d8-4965-b970-a1ce969d1eca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecf4db289fba4e4a806198c62d31f7c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:38.873334 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd5c5d8-a6d8-4965-b970-a1ce969d1eca service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:38.873334 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd5c5d8-a6d8-4965-b970-a1ce969d1eca service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:38.873334 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd5c5d8-a6d8-4965-b970-a1ce969d1eca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:38.880089 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd5c5d8-a6d8-4965-b970-a1ce969d1eca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:38.881623 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd5c5d8-a6d8-4965-b970-a1ce969d1eca service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:38.893776 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1080/2149] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:47:38 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 22:47:38.895166 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd5c5d8-a6d8-4965-b970-a1ce969d1eca service cinder] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:38.897341 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-4cd5c5d8-a6d8-4965-b970-a1ce969d1eca service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:47:38.909937 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1070/2150] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:47:38 2022] PUT /identity/v3/projects/23671aa0492d49eb9749e5dc12643038/users/de57e5e148804737bf51412acc4d7cc0/roles/559942c4de1744169f8b5e0404ecde6b => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 22:47:38.915081 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd5c5d8-a6d8-4965-b970-a1ce969d1eca service cinder] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:38.929218 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e28d04-03b3-492e-a127-b4daedfda1ea admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:38.991795 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7f7e01-c161-446b-8e14-0423a64310ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:38.995849 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7f7e01-c161-446b-8e14-0423a64310ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:38.997088 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7f7e01-c161-446b-8e14-0423a64310ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:38.997381 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7f7e01-c161-446b-8e14-0423a64310ba admin admin] PATH_INFO: `/v3/projects/dc95801c67d14b099cb22a22d7ce7d37` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:38.998739 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7f7e01-c161-446b-8e14-0423a64310ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc95801c67d14b099cb22a22d7ce7d37)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:39.001673 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7f7e01-c161-446b-8e14-0423a64310ba admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:39.020799 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e28d04-03b3-492e-a127-b4daedfda1ea admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:39.026019 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c681c857-ec22-4f38-ae09-7a522e808ef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:39.027072 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c681c857-ec22-4f38-ae09-7a522e808ef1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:39.027336 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c681c857-ec22-4f38-ae09-7a522e808ef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:39.027635 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c681c857-ec22-4f38-ae09-7a522e808ef1 admin admin] PATH_INFO: `/v3/users/95d3a071e4094ffb8057b6ab505b0a62` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:39.028845 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c681c857-ec22-4f38-ae09-7a522e808ef1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=95d3a071e4094ffb8057b6ab505b0a62)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:39.031208 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c681c857-ec22-4f38-ae09-7a522e808ef1 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:39.147461 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-7e7f7e01-c161-446b-8e14-0423a64310ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc95801c67d14b099cb22a22d7ce7d37', 'request_id': 'req-7e7f7e01-c161-446b-8e14-0423a64310ba'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:39.156427 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-7e7f7e01-c161-446b-8e14-0423a64310ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc95801c67d14b099cb22a22d7ce7d37', 'request_id': 'req-7e7f7e01-c161-446b-8e14-0423a64310ba'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:39.210793 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:47:39.227216 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-7e7f7e01-c161-446b-8e14-0423a64310ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc95801c67d14b099cb22a22d7ce7d37', 'request_id': 'req-7e7f7e01-c161-446b-8e14-0423a64310ba'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:39.238858 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:47:39.244530 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-7e7f7e01-c161-446b-8e14-0423a64310ba admin admin] The token cache is being invalidate because project dc95801c67d14b099cb22a22d7ce7d37 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:47:39.244758 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-7e7f7e01-c161-446b-8e14-0423a64310ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:39.246045 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-7e7f7e01-c161-446b-8e14-0423a64310ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:39.247698 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1081/2151] 10.222.0.81 () {58 vars in 1232 bytes} [Wed Jul 20 22:47:38 2022] DELETE /identity/v3/projects/dc95801c67d14b099cb22a22d7ce7d37 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:47:39.250288 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-c681c857-ec22-4f38-ae09-7a522e808ef1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95d3a071e4094ffb8057b6ab505b0a62'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:39.252245 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-c681c857-ec22-4f38-ae09-7a522e808ef1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95d3a071e4094ffb8057b6ab505b0a62'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:39.254778 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5769b62-cbda-42f7-8451-f12999cfaacf None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:39.254929 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-c681c857-ec22-4f38-ae09-7a522e808ef1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95d3a071e4094ffb8057b6ab505b0a62'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:39.255083 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5769b62-cbda-42f7-8451-f12999cfaacf None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:39.255083 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5769b62-cbda-42f7-8451-f12999cfaacf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:39.271843 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-c681c857-ec22-4f38-ae09-7a522e808ef1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95d3a071e4094ffb8057b6ab505b0a62'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:39.301627 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-e5769b62-cbda-42f7-8451-f12999cfaacf None None] MFA Rules not processed for user `de57e5e148804737bf51412acc4d7cc0`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:47:39.302933 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-e5769b62-cbda-42f7-8451-f12999cfaacf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:47:39.330222 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-c681c857-ec22-4f38-ae09-7a522e808ef1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95d3a071e4094ffb8057b6ab505b0a62'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:39.356142 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1071/2152] 10.222.0.81 () {58 vars in 1223 bytes} [Wed Jul 20 22:47:38 2022] DELETE /identity/v3/users/95d3a071e4094ffb8057b6ab505b0a62 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:47:39.357602 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc504dd-03e5-4db2-b880-0ed712763549 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:39.358830 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc504dd-03e5-4db2-b880-0ed712763549 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:39.358830 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc504dd-03e5-4db2-b880-0ed712763549 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:39.378388 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-9cc504dd-03e5-4db2-b880-0ed712763549 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:47:39.379669 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-9cc504dd-03e5-4db2-b880-0ed712763549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:47:39.380194 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1082/2153] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:47:39 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:47:39.381630 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4379b8-dd82-4bf7-82c9-7eee6c25a631 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:39.381936 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4379b8-dd82-4bf7-82c9-7eee6c25a631 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:39.382367 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4379b8-dd82-4bf7-82c9-7eee6c25a631 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:39.400811 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-1b4379b8-dd82-4bf7-82c9-7eee6c25a631 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:47:39.402120 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-1b4379b8-dd82-4bf7-82c9-7eee6c25a631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:47:39.463780 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1072/2154] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:47:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:47:39.468801 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc504dd-03e5-4db2-b880-0ed712763549 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:39.471880 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-9cc504dd-03e5-4db2-b880-0ed712763549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:47:39.489494 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1083/2155] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:47:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:47:39.490402 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97768b4-8deb-4632-bb05-c003c29dd909 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:39.490494 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97768b4-8deb-4632-bb05-c003c29dd909 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:39.490775 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97768b4-8deb-4632-bb05-c003c29dd909 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:39.519161 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-c97768b4-8deb-4632-bb05-c003c29dd909 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:47:39.520330 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-c97768b4-8deb-4632-bb05-c003c29dd909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:47:39.536870 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1084/2156] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:47:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:47:39.536870 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97768b4-8deb-4632-bb05-c003c29dd909 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:39.536870 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-c97768b4-8deb-4632-bb05-c003c29dd909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:47:39.592149 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc504dd-03e5-4db2-b880-0ed712763549 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:39.597281 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990edd16-c273-4f98-8906-d45b8c3a7dcb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecf4db289fba4e4a806198c62d31f7c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:39.598270 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990edd16-c273-4f98-8906-d45b8c3a7dcb service cinder] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:39.598586 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990edd16-c273-4f98-8906-d45b8c3a7dcb service cinder] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:39.599556 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990edd16-c273-4f98-8906-d45b8c3a7dcb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:39.641759 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97768b4-8deb-4632-bb05-c003c29dd909 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:39.652867 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1a5657-03ae-4e63-a2d4-94ebf46c360e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:39.653835 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1a5657-03ae-4e63-a2d4-94ebf46c360e service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:39.653835 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1a5657-03ae-4e63-a2d4-94ebf46c360e service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:39.654324 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1a5657-03ae-4e63-a2d4-94ebf46c360e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:39.669024 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1a5657-03ae-4e63-a2d4-94ebf46c360e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:39.670502 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1a5657-03ae-4e63-a2d4-94ebf46c360e service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:39.689290 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1085/2157] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:47:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:47:39.689634 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1a5657-03ae-4e63-a2d4-94ebf46c360e service neutron] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:39.700216 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1a5657-03ae-4e63-a2d4-94ebf46c360e service neutron] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:39.705957 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990edd16-c273-4f98-8906-d45b8c3a7dcb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:39.707679 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990edd16-c273-4f98-8906-d45b8c3a7dcb service cinder] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:39.708549 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95932d9-9c0e-4be3-bf6d-532ea39ccfef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:39.709240 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95932d9-9c0e-4be3-bf6d-532ea39ccfef admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:39.709533 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95932d9-9c0e-4be3-bf6d-532ea39ccfef admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:39.709828 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95932d9-9c0e-4be3-bf6d-532ea39ccfef admin admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:39.710652 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95932d9-9c0e-4be3-bf6d-532ea39ccfef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:47:39.710970 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95932d9-9c0e-4be3-bf6d-532ea39ccfef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:39.712941 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95932d9-9c0e-4be3-bf6d-532ea39ccfef admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:39.728669 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1086/2158] 10.222.0.81 () {58 vars in 1152 bytes} [Wed Jul 20 22:47:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:47:39.732747 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1073/2159] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:47:39 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:47:39.734225 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95932d9-9c0e-4be3-bf6d-532ea39ccfef admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:39.760531 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95932d9-9c0e-4be3-bf6d-532ea39ccfef admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:39.787227 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:47:39.787623 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c507bffe-11a7-45d3-83be-7ff9707ac689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:39.788720 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c507bffe-11a7-45d3-83be-7ff9707ac689 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:39.789016 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c507bffe-11a7-45d3-83be-7ff9707ac689 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:39.789304 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c507bffe-11a7-45d3-83be-7ff9707ac689 admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:39.790027 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c507bffe-11a7-45d3-83be-7ff9707ac689 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:39.792125 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c507bffe-11a7-45d3-83be-7ff9707ac689 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:39.940804 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1087/2160] 10.222.0.81 () {60 vars in 1152 bytes} [Wed Jul 20 22:47:39 2022] POST /identity/v3/projects => generated 385 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:47:39.944791 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990edd16-c273-4f98-8906-d45b8c3a7dcb service cinder] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:39.957506 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990edd16-c273-4f98-8906-d45b8c3a7dcb service cinder] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:39.965295 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:47:40.052629 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b121a8b6-5273-4619-81c0-8b0389cce83b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:40.053646 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b121a8b6-5273-4619-81c0-8b0389cce83b admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:40.054081 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b121a8b6-5273-4619-81c0-8b0389cce83b admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:40.054287 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b121a8b6-5273-4619-81c0-8b0389cce83b admin admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:40.055101 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b121a8b6-5273-4619-81c0-8b0389cce83b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:40.057369 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b121a8b6-5273-4619-81c0-8b0389cce83b admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:40.163629 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1074/2161] 10.222.0.81 () {60 vars in 1143 bytes} [Wed Jul 20 22:47:39 2022] POST /identity/v3/users => generated 293 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:47:40.164285 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c507bffe-11a7-45d3-83be-7ff9707ac689 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:40.180002 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c507bffe-11a7-45d3-83be-7ff9707ac689 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:40.188031 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e048d47-0d06-4542-843f-bfd45dd64581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:40.188744 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e048d47-0d06-4542-843f-bfd45dd64581 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:40.189034 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e048d47-0d06-4542-843f-bfd45dd64581 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:40.189319 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e048d47-0d06-4542-843f-bfd45dd64581 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:40.190094 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e048d47-0d06-4542-843f-bfd45dd64581 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:40.192024 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e048d47-0d06-4542-843f-bfd45dd64581 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:40.204351 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1088/2162] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:47:40 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 22:47:40.208296 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b121a8b6-5273-4619-81c0-8b0389cce83b admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:40.219790 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b121a8b6-5273-4619-81c0-8b0389cce83b admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:40.224768 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e2ad67-0f64-4958-bc8e-976c93b86c0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:40.225456 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e2ad67-0f64-4958-bc8e-976c93b86c0e admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:40.225744 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e2ad67-0f64-4958-bc8e-976c93b86c0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:40.226025 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e2ad67-0f64-4958-bc8e-976c93b86c0e admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:40.226761 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e2ad67-0f64-4958-bc8e-976c93b86c0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:40.228409 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e2ad67-0f64-4958-bc8e-976c93b86c0e admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:40.239307 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1075/2163] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:47:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:47:40.243314 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e048d47-0d06-4542-843f-bfd45dd64581 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:40.256846 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e048d47-0d06-4542-843f-bfd45dd64581 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:40.266616 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054eff7a-e0ca-4665-9438-ac6f49718ad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:40.267641 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054eff7a-e0ca-4665-9438-ac6f49718ad7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:40.268039 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054eff7a-e0ca-4665-9438-ac6f49718ad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:40.268355 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054eff7a-e0ca-4665-9438-ac6f49718ad7 admin admin] PATH_INFO: `/v3/projects/784b649539a44bbdbc040a34f840bc5d/users/0c13a0dcefff48a499c7acaf396a74d1/roles/559942c4de1744169f8b5e0404ecde6b` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:40.289405 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e2ad67-0f64-4958-bc8e-976c93b86c0e admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:40.291394 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-56e2ad67-0f64-4958-bc8e-976c93b86c0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:47:40.306987 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054eff7a-e0ca-4665-9438-ac6f49718ad7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=784b649539a44bbdbc040a34f840bc5d, user_id=0c13a0dcefff48a499c7acaf396a74d1, role_id=559942c4de1744169f8b5e0404ecde6b)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:40.311133 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054eff7a-e0ca-4665-9438-ac6f49718ad7 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:40.315442 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e2ad67-0f64-4958-bc8e-976c93b86c0e admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:40.326284 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5783b0-b7ae-4b87-9529-3fc07d7b883f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:40.327505 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5783b0-b7ae-4b87-9529-3fc07d7b883f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:40.339773 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5783b0-b7ae-4b87-9529-3fc07d7b883f admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:40.339773 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5783b0-b7ae-4b87-9529-3fc07d7b883f admin admin] PATH_INFO: `/v3/projects/5a9aff967047434097149a856bd17051` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:40.339773 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5783b0-b7ae-4b87-9529-3fc07d7b883f admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a9aff967047434097149a856bd17051)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:40.339773 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5783b0-b7ae-4b87-9529-3fc07d7b883f admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:40.412288 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1089/2164] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:47:40 2022] PUT /identity/v3/projects/784b649539a44bbdbc040a34f840bc5d/users/0c13a0dcefff48a499c7acaf396a74d1/roles/559942c4de1744169f8b5e0404ecde6b => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:47:40.426836 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c5c626-6494-4917-84be-7a7176a01722 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:40.427335 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c5c626-6494-4917-84be-7a7176a01722 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:40.428143 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c5c626-6494-4917-84be-7a7176a01722 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:40.456102 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-e7c5c626-6494-4917-84be-7a7176a01722 None None] MFA Rules not processed for user `0c13a0dcefff48a499c7acaf396a74d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:47:40.457691 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-e7c5c626-6494-4917-84be-7a7176a01722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:47:40.533580 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1090/2165] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:47:40 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:47:40.538295 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c5c626-6494-4917-84be-7a7176a01722 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:40.557939 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-ce5783b0-b7ae-4b87-9529-3fc07d7b883f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a9aff967047434097149a856bd17051', 'request_id': 'req-ce5783b0-b7ae-4b87-9529-3fc07d7b883f'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:40.609686 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c5c626-6494-4917-84be-7a7176a01722 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:40.662062 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-ce5783b0-b7ae-4b87-9529-3fc07d7b883f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a9aff967047434097149a856bd17051', 'request_id': 'req-ce5783b0-b7ae-4b87-9529-3fc07d7b883f'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:40.671733 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-ce5783b0-b7ae-4b87-9529-3fc07d7b883f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a9aff967047434097149a856bd17051', 'request_id': 'req-ce5783b0-b7ae-4b87-9529-3fc07d7b883f'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:40.680468 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f312b3-8b03-461f-aa25-969d1e228a22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:40.682153 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f312b3-8b03-461f-aa25-969d1e228a22 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:40.682671 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f312b3-8b03-461f-aa25-969d1e228a22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:40.683082 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f312b3-8b03-461f-aa25-969d1e228a22 admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:40.683938 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f312b3-8b03-461f-aa25-969d1e228a22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:40.686046 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f312b3-8b03-461f-aa25-969d1e228a22 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:40.689607 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-ce5783b0-b7ae-4b87-9529-3fc07d7b883f admin admin] The token cache is being invalidate because project 5a9aff967047434097149a856bd17051 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:47:40.689981 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-ce5783b0-b7ae-4b87-9529-3fc07d7b883f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:40.690880 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-ce5783b0-b7ae-4b87-9529-3fc07d7b883f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:40.703044 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1076/2166] 10.222.0.81 () {58 vars in 1232 bytes} [Wed Jul 20 22:47:40 2022] DELETE /identity/v3/projects/5a9aff967047434097149a856bd17051 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:47:40.896847 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1091/2167] 10.222.0.81 () {60 vars in 1152 bytes} [Wed Jul 20 22:47:40 2022] POST /identity/v3/projects => generated 385 bytes in 359 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:47:40.901052 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5783b0-b7ae-4b87-9529-3fc07d7b883f admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:40.903169 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-ce5783b0-b7ae-4b87-9529-3fc07d7b883f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:47:40.933900 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5783b0-b7ae-4b87-9529-3fc07d7b883f admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:40.938574 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:47:41.025094 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842884ab-ab71-42e9-8579-a0c6e576a707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:41.025781 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842884ab-ab71-42e9-8579-a0c6e576a707 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:41.026067 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842884ab-ab71-42e9-8579-a0c6e576a707 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:41.026379 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842884ab-ab71-42e9-8579-a0c6e576a707 admin admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:41.027280 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842884ab-ab71-42e9-8579-a0c6e576a707 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:41.029221 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842884ab-ab71-42e9-8579-a0c6e576a707 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:41.192821 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1077/2168] 10.222.0.81 () {60 vars in 1143 bytes} [Wed Jul 20 22:47:40 2022] POST /identity/v3/users => generated 293 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:47:41.196063 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f312b3-8b03-461f-aa25-969d1e228a22 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:41.208586 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f312b3-8b03-461f-aa25-969d1e228a22 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:41.213547 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0918055-b927-48e0-821c-96101839ee88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:41.214202 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0918055-b927-48e0-821c-96101839ee88 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:41.214494 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0918055-b927-48e0-821c-96101839ee88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:41.214775 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0918055-b927-48e0-821c-96101839ee88 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:41.215562 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0918055-b927-48e0-821c-96101839ee88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:41.217144 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0918055-b927-48e0-821c-96101839ee88 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:41.229915 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1092/2169] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:47:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:47:41.232746 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842884ab-ab71-42e9-8579-a0c6e576a707 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:41.242969 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842884ab-ab71-42e9-8579-a0c6e576a707 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:41.248494 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4fc129-7113-436d-8643-8826e8fc3dca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:41.249200 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4fc129-7113-436d-8643-8826e8fc3dca admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:41.249524 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4fc129-7113-436d-8643-8826e8fc3dca admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:41.249823 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4fc129-7113-436d-8643-8826e8fc3dca admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:41.250761 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4fc129-7113-436d-8643-8826e8fc3dca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:41.252451 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4fc129-7113-436d-8643-8826e8fc3dca admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:41.264368 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1078/2170] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:47:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:47:41.272498 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0918055-b927-48e0-821c-96101839ee88 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:41.283736 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0918055-b927-48e0-821c-96101839ee88 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:41.288688 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3511d1-02a0-40be-b843-26072e9239f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:41.289690 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3511d1-02a0-40be-b843-26072e9239f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:41.290044 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3511d1-02a0-40be-b843-26072e9239f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:41.290338 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3511d1-02a0-40be-b843-26072e9239f5 admin admin] PATH_INFO: `/v3/projects/81c0f7a1233146f2b6240fadc2e703db/users/f533c4e7ad65424bb47b8f98cbdd8cde/roles/559942c4de1744169f8b5e0404ecde6b` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:41.307462 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3511d1-02a0-40be-b843-26072e9239f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81c0f7a1233146f2b6240fadc2e703db, user_id=f533c4e7ad65424bb47b8f98cbdd8cde, role_id=559942c4de1744169f8b5e0404ecde6b)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:41.310638 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3511d1-02a0-40be-b843-26072e9239f5 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:41.434363 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:DELETE=1,keystone:SELECT=99 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:47:41.441982 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1093/2171] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:47:41 2022] PUT /identity/v3/projects/81c0f7a1233146f2b6240fadc2e703db/users/f533c4e7ad65424bb47b8f98cbdd8cde/roles/559942c4de1744169f8b5e0404ecde6b => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:47:41.452370 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b03380-e612-42ad-9a10-4c9c2b3873a4 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:41.452667 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b03380-e612-42ad-9a10-4c9c2b3873a4 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:41.452956 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b03380-e612-42ad-9a10-4c9c2b3873a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:41.485430 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-a9b03380-e612-42ad-9a10-4c9c2b3873a4 None None] MFA Rules not processed for user `f533c4e7ad65424bb47b8f98cbdd8cde`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:47:41.486725 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-a9b03380-e612-42ad-9a10-4c9c2b3873a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:47:41.568973 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1079/2172] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:47:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:47:41.577322 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3511d1-02a0-40be-b843-26072e9239f5 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:41.646082 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3511d1-02a0-40be-b843-26072e9239f5 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:41.650942 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c25a719-3afa-490e-ad03-b88103c6ba73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:41.651943 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c25a719-3afa-490e-ad03-b88103c6ba73 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:41.652200 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c25a719-3afa-490e-ad03-b88103c6ba73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:41.652486 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c25a719-3afa-490e-ad03-b88103c6ba73 admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:41.653190 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c25a719-3afa-490e-ad03-b88103c6ba73 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:41.655126 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c25a719-3afa-490e-ad03-b88103c6ba73 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:41.796999 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1094/2173] 10.222.0.81 () {60 vars in 1152 bytes} [Wed Jul 20 22:47:41 2022] POST /identity/v3/projects => generated 383 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:47:41.800025 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b03380-e612-42ad-9a10-4c9c2b3873a4 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:41.839893 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b03380-e612-42ad-9a10-4c9c2b3873a4 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:41.925628 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1afc190-0243-48be-9aaf-e2c85af51f77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:41.926441 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1afc190-0243-48be-9aaf-e2c85af51f77 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:41.926835 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1afc190-0243-48be-9aaf-e2c85af51f77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:41.927344 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1afc190-0243-48be-9aaf-e2c85af51f77 admin admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:41.928361 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1afc190-0243-48be-9aaf-e2c85af51f77 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:41.930414 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1afc190-0243-48be-9aaf-e2c85af51f77 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:42.083195 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c25a719-3afa-490e-ad03-b88103c6ba73 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:42.085566 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1080/2174] 10.222.0.81 () {60 vars in 1143 bytes} [Wed Jul 20 22:47:41 2022] POST /identity/v3/users => generated 298 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 22:47:42.099036 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c25a719-3afa-490e-ad03-b88103c6ba73 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:42.111577 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec1ad4b-c1ce-41c3-a7b0-dd009b47be10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:42.111577 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec1ad4b-c1ce-41c3-a7b0-dd009b47be10 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:42.111577 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec1ad4b-c1ce-41c3-a7b0-dd009b47be10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:42.111577 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec1ad4b-c1ce-41c3-a7b0-dd009b47be10 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:42.112006 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec1ad4b-c1ce-41c3-a7b0-dd009b47be10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:42.112006 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec1ad4b-c1ce-41c3-a7b0-dd009b47be10 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:42.123541 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1095/2175] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:47:42 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 22:47:42.129248 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1afc190-0243-48be-9aaf-e2c85af51f77 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:42.142291 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:47:42.146190 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1afc190-0243-48be-9aaf-e2c85af51f77 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:42.154053 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e57ca29-e684-4823-b56a-303765754ace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:42.156143 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e57ca29-e684-4823-b56a-303765754ace admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:42.156504 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e57ca29-e684-4823-b56a-303765754ace admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:42.156813 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e57ca29-e684-4823-b56a-303765754ace admin admin] PATH_INFO: `/v3/projects/acebca96d1a24a02af045a788529530c/users/8828bff2fe7e4cfeabd346c8d3d73a9e/roles/d89b6c92d35a401a8517f1d47f6be843` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:42.173863 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e57ca29-e684-4823-b56a-303765754ace admin admin] RBAC: Authorizing `identity:create_grant(project_id=acebca96d1a24a02af045a788529530c, user_id=8828bff2fe7e4cfeabd346c8d3d73a9e, role_id=d89b6c92d35a401a8517f1d47f6be843)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:42.176631 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e57ca29-e684-4823-b56a-303765754ace admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:42.302552 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1081/2176] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:47:42 2022] PUT /identity/v3/projects/acebca96d1a24a02af045a788529530c/users/8828bff2fe7e4cfeabd346c8d3d73a9e/roles/d89b6c92d35a401a8517f1d47f6be843 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:47:42.306441 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec1ad4b-c1ce-41c3-a7b0-dd009b47be10 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:42.318550 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec1ad4b-c1ce-41c3-a7b0-dd009b47be10 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:42.375958 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:47:42.429772 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ba5dfb-ef11-4ad4-bb32-1b1f70f8e439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:42.429772 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ba5dfb-ef11-4ad4-bb32-1b1f70f8e439 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:42.429772 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ba5dfb-ef11-4ad4-bb32-1b1f70f8e439 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:42.429772 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ba5dfb-ef11-4ad4-bb32-1b1f70f8e439 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:42.430504 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ba5dfb-ef11-4ad4-bb32-1b1f70f8e439 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:42.432149 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ba5dfb-ef11-4ad4-bb32-1b1f70f8e439 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:42.445347 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1096/2177] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:47:42 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:47:42.458297 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e57ca29-e684-4823-b56a-303765754ace admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:42.466267 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e57ca29-e684-4823-b56a-303765754ace admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:42.472074 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84159852-927c-4f20-8c44-4da11109d259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:42.473500 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84159852-927c-4f20-8c44-4da11109d259 admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:42.473982 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84159852-927c-4f20-8c44-4da11109d259 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:42.474432 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84159852-927c-4f20-8c44-4da11109d259 admin admin] PATH_INFO: `/v3/projects/acebca96d1a24a02af045a788529530c/users/8828bff2fe7e4cfeabd346c8d3d73a9e/roles/d89b6c92d35a401a8517f1d47f6be843` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:42.476889 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84159852-927c-4f20-8c44-4da11109d259 admin admin] RBAC: Authorizing `identity:create_grant(project_id=acebca96d1a24a02af045a788529530c, user_id=8828bff2fe7e4cfeabd346c8d3d73a9e, role_id=d89b6c92d35a401a8517f1d47f6be843)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:42.480228 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84159852-927c-4f20-8c44-4da11109d259 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:42.491811 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1082/2178] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:47:42 2022] PUT /identity/v3/projects/acebca96d1a24a02af045a788529530c/users/8828bff2fe7e4cfeabd346c8d3d73a9e/roles/d89b6c92d35a401a8517f1d47f6be843 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:47:42.505372 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fd5b03-7057-44f5-a092-50260f06de00 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:42.506076 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fd5b03-7057-44f5-a092-50260f06de00 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:42.506566 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fd5b03-7057-44f5-a092-50260f06de00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:42.535830 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-49fd5b03-7057-44f5-a092-50260f06de00 None None] MFA Rules not processed for user `8828bff2fe7e4cfeabd346c8d3d73a9e`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:47:42.537063 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-49fd5b03-7057-44f5-a092-50260f06de00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:47:42.566158 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:47:42.598529 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:47:42.621248 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1097/2179] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:47:42 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:47:42.633685 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84159852-927c-4f20-8c44-4da11109d259 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:42.635432 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-84159852-927c-4f20-8c44-4da11109d259 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:47:42.697933 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84159852-927c-4f20-8c44-4da11109d259 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:42.702239 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c93b9bb-864b-4c8c-a94d-c9aa9bc18903 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecf4db289fba4e4a806198c62d31f7c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:42.702899 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c93b9bb-864b-4c8c-a94d-c9aa9bc18903 service cinder] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:42.703219 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c93b9bb-864b-4c8c-a94d-c9aa9bc18903 service cinder] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:42.703504 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c93b9bb-864b-4c8c-a94d-c9aa9bc18903 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:42.705427 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-4c93b9bb-864b-4c8c-a94d-c9aa9bc18903 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:47:42.767832 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c93b9bb-864b-4c8c-a94d-c9aa9bc18903 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:42.769310 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c93b9bb-864b-4c8c-a94d-c9aa9bc18903 service cinder] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:42.778683 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1083/2180] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:47:42 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 22:47:42.836901 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fd5b03-7057-44f5-a092-50260f06de00 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:42.838563 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-49fd5b03-7057-44f5-a092-50260f06de00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:47:42.910588 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fd5b03-7057-44f5-a092-50260f06de00 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:42.915845 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312ba494-9ed4-4759-b615-bd014b07bd8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:42.916819 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312ba494-9ed4-4759-b615-bd014b07bd8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:42.917864 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312ba494-9ed4-4759-b615-bd014b07bd8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:42.918220 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312ba494-9ed4-4759-b615-bd014b07bd8a admin admin] PATH_INFO: `/v3/users/1b458e3837be425bba1f880895918287` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:42.919533 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312ba494-9ed4-4759-b615-bd014b07bd8a admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b458e3837be425bba1f880895918287)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:42.921519 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312ba494-9ed4-4759-b615-bd014b07bd8a admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:42.925596 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:47:43.016130 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:47:43.111884 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-312ba494-9ed4-4759-b615-bd014b07bd8a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b458e3837be425bba1f880895918287'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:43.112591 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-312ba494-9ed4-4759-b615-bd014b07bd8a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b458e3837be425bba1f880895918287'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:43.113202 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-312ba494-9ed4-4759-b615-bd014b07bd8a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b458e3837be425bba1f880895918287'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:43.120180 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-312ba494-9ed4-4759-b615-bd014b07bd8a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b458e3837be425bba1f880895918287'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:43.272718 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-312ba494-9ed4-4759-b615-bd014b07bd8a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b458e3837be425bba1f880895918287'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:43.292873 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1098/2181] 10.222.0.81 () {58 vars in 1223 bytes} [Wed Jul 20 22:47:42 2022] DELETE /identity/v3/users/1b458e3837be425bba1f880895918287 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:47:43.607392 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c93b9bb-864b-4c8c-a94d-c9aa9bc18903 service cinder] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:43.636508 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c93b9bb-864b-4c8c-a94d-c9aa9bc18903 service cinder] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:43.701171 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7e7b5d-2666-48b5-bc4a-f4305c98e1a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:43.702060 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7e7b5d-2666-48b5-bc4a-f4305c98e1a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:43.702350 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7e7b5d-2666-48b5-bc4a-f4305c98e1a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:43.702633 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7e7b5d-2666-48b5-bc4a-f4305c98e1a7 admin admin] PATH_INFO: `/v3/projects/788c1d44ad5d442e8ec58f1d76ae649b` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:43.704029 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7e7b5d-2666-48b5-bc4a-f4305c98e1a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=788c1d44ad5d442e8ec58f1d76ae649b)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:43.705953 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7e7b5d-2666-48b5-bc4a-f4305c98e1a7 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:43.849310 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-1c7e7b5d-2666-48b5-bc4a-f4305c98e1a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '788c1d44ad5d442e8ec58f1d76ae649b', 'request_id': 'req-1c7e7b5d-2666-48b5-bc4a-f4305c98e1a7'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:43.947452 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-1c7e7b5d-2666-48b5-bc4a-f4305c98e1a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '788c1d44ad5d442e8ec58f1d76ae649b', 'request_id': 'req-1c7e7b5d-2666-48b5-bc4a-f4305c98e1a7'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:43.956749 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-1c7e7b5d-2666-48b5-bc4a-f4305c98e1a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '788c1d44ad5d442e8ec58f1d76ae649b', 'request_id': 'req-1c7e7b5d-2666-48b5-bc4a-f4305c98e1a7'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:43.968656 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-1c7e7b5d-2666-48b5-bc4a-f4305c98e1a7 admin admin] The token cache is being invalidate because project 788c1d44ad5d442e8ec58f1d76ae649b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:47:43.968943 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-1c7e7b5d-2666-48b5-bc4a-f4305c98e1a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:43.969606 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-1c7e7b5d-2666-48b5-bc4a-f4305c98e1a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:47:43.971385 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1084/2182] 10.222.0.81 () {58 vars in 1232 bytes} [Wed Jul 20 22:47:43 2022] DELETE /identity/v3/projects/788c1d44ad5d442e8ec58f1d76ae649b => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:47:43.976363 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6efb5ea-a3a5-4fd3-954a-ead3896a6115 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:43.976830 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6efb5ea-a3a5-4fd3-954a-ead3896a6115 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:43.976938 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6efb5ea-a3a5-4fd3-954a-ead3896a6115 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:43.995734 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-b6efb5ea-a3a5-4fd3-954a-ead3896a6115 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:47:43.997031 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-b6efb5ea-a3a5-4fd3-954a-ead3896a6115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:47:44.050866 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:47:44.058643 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1099/2183] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:47:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:47:44.066782 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e025d4cb-a899-458d-9300-f549fd689546 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:44.067149 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e025d4cb-a899-458d-9300-f549fd689546 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:44.067459 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e025d4cb-a899-458d-9300-f549fd689546 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:44.088631 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:47:44.095771 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-e025d4cb-a899-458d-9300-f549fd689546 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:47:44.095771 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-e025d4cb-a899-458d-9300-f549fd689546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:47:44.102806 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1085/2184] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:47:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:47:44.107271 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6efb5ea-a3a5-4fd3-954a-ead3896a6115 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:44.117694 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6efb5ea-a3a5-4fd3-954a-ead3896a6115 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:44.123605 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00e1009-6aa9-47bf-b41e-21279310926e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:44.124299 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00e1009-6aa9-47bf-b41e-21279310926e admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:44.125018 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00e1009-6aa9-47bf-b41e-21279310926e admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:44.125371 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00e1009-6aa9-47bf-b41e-21279310926e admin admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:44.126183 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00e1009-6aa9-47bf-b41e-21279310926e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:47:44.126492 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00e1009-6aa9-47bf-b41e-21279310926e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:44.128120 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00e1009-6aa9-47bf-b41e-21279310926e admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:44.141307 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1100/2185] 10.222.0.81 () {58 vars in 1152 bytes} [Wed Jul 20 22:47:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:47:44.146044 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e025d4cb-a899-458d-9300-f549fd689546 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:44.156371 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e025d4cb-a899-458d-9300-f549fd689546 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:44.161443 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b606650c-df5c-4ded-b0f1-9b14b8ae8d1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:44.162277 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b606650c-df5c-4ded-b0f1-9b14b8ae8d1a admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:44.162654 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b606650c-df5c-4ded-b0f1-9b14b8ae8d1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:44.163105 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b606650c-df5c-4ded-b0f1-9b14b8ae8d1a admin admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:44.164989 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b606650c-df5c-4ded-b0f1-9b14b8ae8d1a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:44.166001 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b606650c-df5c-4ded-b0f1-9b14b8ae8d1a admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:44.282969 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1086/2186] 10.222.0.81 () {60 vars in 1152 bytes} [Wed Jul 20 22:47:44 2022] POST /identity/v3/projects => generated 367 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:47:44.284260 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00e1009-6aa9-47bf-b41e-21279310926e admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:44.295475 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00e1009-6aa9-47bf-b41e-21279310926e admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:44.356717 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f98e0b3-4030-4267-998f-042e07f5ae2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:44.357431 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f98e0b3-4030-4267-998f-042e07f5ae2d admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:44.357688 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f98e0b3-4030-4267-998f-042e07f5ae2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:44.357993 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f98e0b3-4030-4267-998f-042e07f5ae2d admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:44.358725 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f98e0b3-4030-4267-998f-042e07f5ae2d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:44.360705 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f98e0b3-4030-4267-998f-042e07f5ae2d admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:44.461934 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1101/2187] 10.222.0.81 () {60 vars in 1143 bytes} [Wed Jul 20 22:47:44 2022] POST /identity/v3/users => generated 284 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:47:44.466105 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b606650c-df5c-4ded-b0f1-9b14b8ae8d1a admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:44.472426 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:47:44.476995 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b606650c-df5c-4ded-b0f1-9b14b8ae8d1a admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:44.482356 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eedb963-9e64-48ca-80a7-684f3a51da9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:44.483067 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eedb963-9e64-48ca-80a7-684f3a51da9d admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:44.483422 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eedb963-9e64-48ca-80a7-684f3a51da9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:44.483699 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eedb963-9e64-48ca-80a7-684f3a51da9d admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:44.484456 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eedb963-9e64-48ca-80a7-684f3a51da9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:44.486037 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eedb963-9e64-48ca-80a7-684f3a51da9d admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:44.497515 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1087/2188] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:47:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 22:47:44.501522 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f98e0b3-4030-4267-998f-042e07f5ae2d admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:44.511637 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f98e0b3-4030-4267-998f-042e07f5ae2d admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:44.516576 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31565722-fa69-4e3c-8c85-94326a70cfeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:44.517248 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31565722-fa69-4e3c-8c85-94326a70cfeb admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:44.517537 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31565722-fa69-4e3c-8c85-94326a70cfeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:44.517835 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31565722-fa69-4e3c-8c85-94326a70cfeb admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:44.518579 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31565722-fa69-4e3c-8c85-94326a70cfeb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:44.520204 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31565722-fa69-4e3c-8c85-94326a70cfeb admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:44.529890 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1102/2189] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:47:44 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:47:44.534044 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eedb963-9e64-48ca-80a7-684f3a51da9d admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:44.544740 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eedb963-9e64-48ca-80a7-684f3a51da9d admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:44.550121 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751f1fe1-7966-4809-baea-af300766b945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:44.551126 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751f1fe1-7966-4809-baea-af300766b945 admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:44.551416 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751f1fe1-7966-4809-baea-af300766b945 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:44.557575 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751f1fe1-7966-4809-baea-af300766b945 admin admin] PATH_INFO: `/v3/projects/19c2bac554864487b87772c8d8663a3d/users/2b09f9a730724f4ebc3415be7a510fac/roles/559942c4de1744169f8b5e0404ecde6b` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:44.568267 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751f1fe1-7966-4809-baea-af300766b945 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19c2bac554864487b87772c8d8663a3d, user_id=2b09f9a730724f4ebc3415be7a510fac, role_id=559942c4de1744169f8b5e0404ecde6b)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:44.571085 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751f1fe1-7966-4809-baea-af300766b945 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:44.695666 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1088/2190] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:47:44 2022] PUT /identity/v3/projects/19c2bac554864487b87772c8d8663a3d/users/2b09f9a730724f4ebc3415be7a510fac/roles/559942c4de1744169f8b5e0404ecde6b => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:47:44.705940 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cf6213-4899-4b29-a593-c67f318d7791 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:44.706449 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cf6213-4899-4b29-a593-c67f318d7791 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:44.706842 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cf6213-4899-4b29-a593-c67f318d7791 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:44.707128 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:47:44.735148 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-78cf6213-4899-4b29-a593-c67f318d7791 None None] MFA Rules not processed for user `2b09f9a730724f4ebc3415be7a510fac`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:47:44.736359 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-78cf6213-4899-4b29-a593-c67f318d7791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:47:44.803756 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1103/2191] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:47:44 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:47:44.814425 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751f1fe1-7966-4809-baea-af300766b945 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:44.816214 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-751f1fe1-7966-4809-baea-af300766b945 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:47:44.827285 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:47:44.875324 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751f1fe1-7966-4809-baea-af300766b945 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:44.879611 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eede5809-0fac-49b3-9ecc-81ea3bafeb94 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecf4db289fba4e4a806198c62d31f7c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:44.880279 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eede5809-0fac-49b3-9ecc-81ea3bafeb94 service cinder] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:44.880568 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eede5809-0fac-49b3-9ecc-81ea3bafeb94 service cinder] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:44.880887 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eede5809-0fac-49b3-9ecc-81ea3bafeb94 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:44.889624 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eede5809-0fac-49b3-9ecc-81ea3bafeb94 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:44.891103 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eede5809-0fac-49b3-9ecc-81ea3bafeb94 service cinder] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:44.899897 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1089/2192] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:47:44 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:47:45.072846 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:47:45.198804 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:47:45.903446 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cf6213-4899-4b29-a593-c67f318d7791 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:45.904874 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-78cf6213-4899-4b29-a593-c67f318d7791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:47:45.917185 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cf6213-4899-4b29-a593-c67f318d7791 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:45.921362 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7ea27f-94af-4dc1-a448-1cfac6d9460e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecf4db289fba4e4a806198c62d31f7c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:45.922029 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7ea27f-94af-4dc1-a448-1cfac6d9460e service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:45.923336 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7ea27f-94af-4dc1-a448-1cfac6d9460e service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:45.923336 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7ea27f-94af-4dc1-a448-1cfac6d9460e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:45.924758 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-6f7ea27f-94af-4dc1-a448-1cfac6d9460e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:47:45.987917 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7ea27f-94af-4dc1-a448-1cfac6d9460e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:45.987917 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7ea27f-94af-4dc1-a448-1cfac6d9460e service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:45.995018 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1104/2193] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:47:45 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:47:47.707377 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eede5809-0fac-49b3-9ecc-81ea3bafeb94 service cinder] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:47:47.717253 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eede5809-0fac-49b3-9ecc-81ea3bafeb94 service cinder] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:47:47.721361 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854e6d77-718a-4a8d-9bd7-d79db147b02b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecf4db289fba4e4a806198c62d31f7c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:47:47.722132 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854e6d77-718a-4a8d-9bd7-d79db147b02b service cinder] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:47:47.722413 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854e6d77-718a-4a8d-9bd7-d79db147b02b service cinder] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:47:47.722693 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854e6d77-718a-4a8d-9bd7-d79db147b02b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:47:47.724584 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-854e6d77-718a-4a8d-9bd7-d79db147b02b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:47:47.792451 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:47:47.809336 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854e6d77-718a-4a8d-9bd7-d79db147b02b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:47:47.811283 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854e6d77-718a-4a8d-9bd7-d79db147b02b service cinder] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:47:47.825239 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1090/2194] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:47:47 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:47:56.095308 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=51 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:47:58.010108 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=4 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:48:02.490969 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7ea27f-94af-4dc1-a448-1cfac6d9460e service cinder] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:48:02.492501 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-6f7ea27f-94af-4dc1-a448-1cfac6d9460e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:48:02.576062 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7ea27f-94af-4dc1-a448-1cfac6d9460e service cinder] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:48:02.581798 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc3c7ed-47f9-4e62-97c2-917e3ad25047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:48:02.582699 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc3c7ed-47f9-4e62-97c2-917e3ad25047 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:48:02.587291 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc3c7ed-47f9-4e62-97c2-917e3ad25047 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:48:02.587291 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc3c7ed-47f9-4e62-97c2-917e3ad25047 admin admin] PATH_INFO: `/v3/users/0c13a0dcefff48a499c7acaf396a74d1` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:48:02.587291 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc3c7ed-47f9-4e62-97c2-917e3ad25047 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c13a0dcefff48a499c7acaf396a74d1)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:48:02.587291 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc3c7ed-47f9-4e62-97c2-917e3ad25047 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:48:02.805158 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-7cc3c7ed-47f9-4e62-97c2-917e3ad25047 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c13a0dcefff48a499c7acaf396a74d1'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:48:02.805823 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-7cc3c7ed-47f9-4e62-97c2-917e3ad25047 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c13a0dcefff48a499c7acaf396a74d1'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:48:02.806467 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-7cc3c7ed-47f9-4e62-97c2-917e3ad25047 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c13a0dcefff48a499c7acaf396a74d1'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:48:02.814116 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-7cc3c7ed-47f9-4e62-97c2-917e3ad25047 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c13a0dcefff48a499c7acaf396a74d1'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:48:02.899838 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-7cc3c7ed-47f9-4e62-97c2-917e3ad25047 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c13a0dcefff48a499c7acaf396a74d1'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:48:02.918326 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1105/2195] 10.222.0.81 () {58 vars in 1223 bytes} [Wed Jul 20 22:48:02 2022] DELETE /identity/v3/users/0c13a0dcefff48a499c7acaf396a74d1 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 22:48:02.922595 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ded5c56-04ae-43e5-90cf-eb0807183e67 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:48:02.923095 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ded5c56-04ae-43e5-90cf-eb0807183e67 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:48:02.923475 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ded5c56-04ae-43e5-90cf-eb0807183e67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:48:02.943226 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-0ded5c56-04ae-43e5-90cf-eb0807183e67 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:48:02.943704 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-0ded5c56-04ae-43e5-90cf-eb0807183e67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:48:03.006144 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1091/2196] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:48:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:48:03.016093 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc3c7ed-47f9-4e62-97c2-917e3ad25047 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:48:03.017540 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-7cc3c7ed-47f9-4e62-97c2-917e3ad25047 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:48:03.078811 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc3c7ed-47f9-4e62-97c2-917e3ad25047 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:48:03.082620 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33ff00d-2302-406d-8090-91d6832599a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:48:03.083327 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33ff00d-2302-406d-8090-91d6832599a5 service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:48:03.083638 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33ff00d-2302-406d-8090-91d6832599a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:48:03.083911 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33ff00d-2302-406d-8090-91d6832599a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:48:03.093236 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33ff00d-2302-406d-8090-91d6832599a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:48:03.094710 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33ff00d-2302-406d-8090-91d6832599a5 service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:48:03.104100 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1106/2197] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:48:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 22:48:03.579401 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ded5c56-04ae-43e5-90cf-eb0807183e67 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:48:03.579894 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-0ded5c56-04ae-43e5-90cf-eb0807183e67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:48:03.595566 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ded5c56-04ae-43e5-90cf-eb0807183e67 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:48:03.602955 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825bf41e-a2e5-4009-a8da-24be3fab8d58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:48:03.603881 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825bf41e-a2e5-4009-a8da-24be3fab8d58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:48:03.604168 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825bf41e-a2e5-4009-a8da-24be3fab8d58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:48:03.604445 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825bf41e-a2e5-4009-a8da-24be3fab8d58 admin admin] PATH_INFO: `/v3/projects/784b649539a44bbdbc040a34f840bc5d` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:48:03.617197 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825bf41e-a2e5-4009-a8da-24be3fab8d58 admin admin] RBAC: Authorizing `identity:delete_project(project_id=784b649539a44bbdbc040a34f840bc5d)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:48:03.619200 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825bf41e-a2e5-4009-a8da-24be3fab8d58 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:48:03.749114 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-825bf41e-a2e5-4009-a8da-24be3fab8d58 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '784b649539a44bbdbc040a34f840bc5d', 'request_id': 'req-825bf41e-a2e5-4009-a8da-24be3fab8d58'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:48:03.825882 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-825bf41e-a2e5-4009-a8da-24be3fab8d58 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '784b649539a44bbdbc040a34f840bc5d', 'request_id': 'req-825bf41e-a2e5-4009-a8da-24be3fab8d58'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:48:03.832515 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-825bf41e-a2e5-4009-a8da-24be3fab8d58 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '784b649539a44bbdbc040a34f840bc5d', 'request_id': 'req-825bf41e-a2e5-4009-a8da-24be3fab8d58'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:48:03.843899 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-825bf41e-a2e5-4009-a8da-24be3fab8d58 admin admin] The token cache is being invalidate because project 784b649539a44bbdbc040a34f840bc5d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:48:03.844192 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-825bf41e-a2e5-4009-a8da-24be3fab8d58 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:48:03.844832 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-825bf41e-a2e5-4009-a8da-24be3fab8d58 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:48:03.846372 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1092/2198] 10.222.0.81 () {58 vars in 1232 bytes} [Wed Jul 20 22:48:03 2022] DELETE /identity/v3/projects/784b649539a44bbdbc040a34f840bc5d => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:48:03.850983 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33ff00d-2302-406d-8090-91d6832599a5 service neutron] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:48:03.860569 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33ff00d-2302-406d-8090-91d6832599a5 service neutron] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:48:03.878617 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:48:03.918485 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac6ad93-79b0-47ff-9a48-a27e7cac727b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:48:03.919498 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac6ad93-79b0-47ff-9a48-a27e7cac727b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:48:03.919791 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac6ad93-79b0-47ff-9a48-a27e7cac727b admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:48:03.920073 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac6ad93-79b0-47ff-9a48-a27e7cac727b admin admin] PATH_INFO: `/v3/users/f533c4e7ad65424bb47b8f98cbdd8cde` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:48:03.921400 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac6ad93-79b0-47ff-9a48-a27e7cac727b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f533c4e7ad65424bb47b8f98cbdd8cde)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:48:03.923589 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac6ad93-79b0-47ff-9a48-a27e7cac727b admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:48:03.935421 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7c03bd-4b69-431f-9d26-d51c527c8c8f None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:48:03.935951 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7c03bd-4b69-431f-9d26-d51c527c8c8f None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:48:03.936030 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7c03bd-4b69-431f-9d26-d51c527c8c8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:48:03.953502 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-8d7c03bd-4b69-431f-9d26-d51c527c8c8f None None] MFA Rules not processed for user `1ed9249b2a9b42d9972ed71c04f2c888`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:48:03.954712 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-8d7c03bd-4b69-431f-9d26-d51c527c8c8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:48:04.010258 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1093/2199] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:48:03 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 22:48:04.240918 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-6ac6ad93-79b0-47ff-9a48-a27e7cac727b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f533c4e7ad65424bb47b8f98cbdd8cde'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:48:04.241610 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-6ac6ad93-79b0-47ff-9a48-a27e7cac727b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f533c4e7ad65424bb47b8f98cbdd8cde'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:48:04.242228 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-6ac6ad93-79b0-47ff-9a48-a27e7cac727b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f533c4e7ad65424bb47b8f98cbdd8cde'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:48:04.250233 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-6ac6ad93-79b0-47ff-9a48-a27e7cac727b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f533c4e7ad65424bb47b8f98cbdd8cde'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:48:04.334486 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-6ac6ad93-79b0-47ff-9a48-a27e7cac727b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f533c4e7ad65424bb47b8f98cbdd8cde'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:48:04.352390 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1107/2200] 10.222.0.81 () {58 vars in 1223 bytes} [Wed Jul 20 22:48:03 2022] DELETE /identity/v3/users/f533c4e7ad65424bb47b8f98cbdd8cde => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:48:04.798825 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7c03bd-4b69-431f-9d26-d51c527c8c8f None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:48:04.801387 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-8d7c03bd-4b69-431f-9d26-d51c527c8c8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:48:04.817082 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7c03bd-4b69-431f-9d26-d51c527c8c8f None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:48:04.850170 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:48:04.881629 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf0edb6-e4cd-4daf-bc25-099176bdaec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:48:04.882631 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf0edb6-e4cd-4daf-bc25-099176bdaec4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:48:04.882955 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf0edb6-e4cd-4daf-bc25-099176bdaec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:48:04.889344 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf0edb6-e4cd-4daf-bc25-099176bdaec4 admin admin] PATH_INFO: `/v3/projects/81c0f7a1233146f2b6240fadc2e703db` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:48:04.889344 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf0edb6-e4cd-4daf-bc25-099176bdaec4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=81c0f7a1233146f2b6240fadc2e703db)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:48:04.889344 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf0edb6-e4cd-4daf-bc25-099176bdaec4 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:48:05.004842 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-2bf0edb6-e4cd-4daf-bc25-099176bdaec4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81c0f7a1233146f2b6240fadc2e703db', 'request_id': 'req-2bf0edb6-e4cd-4daf-bc25-099176bdaec4'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:48:05.060954 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-2bf0edb6-e4cd-4daf-bc25-099176bdaec4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81c0f7a1233146f2b6240fadc2e703db', 'request_id': 'req-2bf0edb6-e4cd-4daf-bc25-099176bdaec4'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:48:05.067971 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-2bf0edb6-e4cd-4daf-bc25-099176bdaec4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81c0f7a1233146f2b6240fadc2e703db', 'request_id': 'req-2bf0edb6-e4cd-4daf-bc25-099176bdaec4'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:48:05.081959 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-2bf0edb6-e4cd-4daf-bc25-099176bdaec4 admin admin] The token cache is being invalidate because project 81c0f7a1233146f2b6240fadc2e703db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:48:05.082596 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-2bf0edb6-e4cd-4daf-bc25-099176bdaec4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:48:05.083177 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-2bf0edb6-e4cd-4daf-bc25-099176bdaec4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:48:05.088010 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1094/2201] 10.222.0.81 () {58 vars in 1232 bytes} [Wed Jul 20 22:48:04 2022] DELETE /identity/v3/projects/81c0f7a1233146f2b6240fadc2e703db => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:48:05.089880 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac6ad93-79b0-47ff-9a48-a27e7cac727b admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:48:05.091512 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-6ac6ad93-79b0-47ff-9a48-a27e7cac727b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:48:05.104996 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac6ad93-79b0-47ff-9a48-a27e7cac727b admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:48:05.163032 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9018f8-4042-45f7-9b26-e29c0f15dd1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:48:05.164220 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9018f8-4042-45f7-9b26-e29c0f15dd1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:48:05.164514 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9018f8-4042-45f7-9b26-e29c0f15dd1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:48:05.164794 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9018f8-4042-45f7-9b26-e29c0f15dd1d admin admin] PATH_INFO: `/v3/users/8828bff2fe7e4cfeabd346c8d3d73a9e` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:48:05.166428 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9018f8-4042-45f7-9b26-e29c0f15dd1d admin admin] RBAC: Authorizing `identity:delete_user(user_id=8828bff2fe7e4cfeabd346c8d3d73a9e)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:48:05.168974 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9018f8-4042-45f7-9b26-e29c0f15dd1d admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:48:05.485090 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:48:05.525051 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-2f9018f8-4042-45f7-9b26-e29c0f15dd1d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8828bff2fe7e4cfeabd346c8d3d73a9e'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:48:05.525051 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-2f9018f8-4042-45f7-9b26-e29c0f15dd1d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8828bff2fe7e4cfeabd346c8d3d73a9e'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:48:05.525051 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-2f9018f8-4042-45f7-9b26-e29c0f15dd1d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8828bff2fe7e4cfeabd346c8d3d73a9e'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:48:05.525051 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-2f9018f8-4042-45f7-9b26-e29c0f15dd1d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8828bff2fe7e4cfeabd346c8d3d73a9e'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:48:05.866108 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-2f9018f8-4042-45f7-9b26-e29c0f15dd1d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8828bff2fe7e4cfeabd346c8d3d73a9e'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:48:05.883766 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1108/2202] 10.222.0.81 () {58 vars in 1223 bytes} [Wed Jul 20 22:48:05 2022] DELETE /identity/v3/users/8828bff2fe7e4cfeabd346c8d3d73a9e => generated 0 bytes in 795 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:48:06.364905 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf0edb6-e4cd-4daf-bc25-099176bdaec4 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:48:06.366412 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-2bf0edb6-e4cd-4daf-bc25-099176bdaec4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:48:06.387433 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf0edb6-e4cd-4daf-bc25-099176bdaec4 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:48:06.443363 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c494ae7d-a367-4b0b-9ff9-d2d04f63c466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:48:06.444291 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c494ae7d-a367-4b0b-9ff9-d2d04f63c466 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:48:06.444620 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c494ae7d-a367-4b0b-9ff9-d2d04f63c466 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:48:06.444900 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c494ae7d-a367-4b0b-9ff9-d2d04f63c466 admin admin] PATH_INFO: `/v3/projects/acebca96d1a24a02af045a788529530c` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:48:06.446100 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c494ae7d-a367-4b0b-9ff9-d2d04f63c466 admin admin] RBAC: Authorizing `identity:delete_project(project_id=acebca96d1a24a02af045a788529530c)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:48:06.448168 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c494ae7d-a367-4b0b-9ff9-d2d04f63c466 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:48:06.557890 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-c494ae7d-a367-4b0b-9ff9-d2d04f63c466 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'acebca96d1a24a02af045a788529530c', 'request_id': 'req-c494ae7d-a367-4b0b-9ff9-d2d04f63c466'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:48:06.610154 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-c494ae7d-a367-4b0b-9ff9-d2d04f63c466 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'acebca96d1a24a02af045a788529530c', 'request_id': 'req-c494ae7d-a367-4b0b-9ff9-d2d04f63c466'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:48:06.618616 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:48:06.618616 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-c494ae7d-a367-4b0b-9ff9-d2d04f63c466 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'acebca96d1a24a02af045a788529530c', 'request_id': 'req-c494ae7d-a367-4b0b-9ff9-d2d04f63c466'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:48:06.628953 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-c494ae7d-a367-4b0b-9ff9-d2d04f63c466 admin admin] The token cache is being invalidate because project acebca96d1a24a02af045a788529530c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:48:06.629180 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-c494ae7d-a367-4b0b-9ff9-d2d04f63c466 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:48:06.629827 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-c494ae7d-a367-4b0b-9ff9-d2d04f63c466 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:48:06.631556 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1095/2203] 10.222.0.81 () {58 vars in 1232 bytes} [Wed Jul 20 22:48:06 2022] DELETE /identity/v3/projects/acebca96d1a24a02af045a788529530c => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:48:06.636438 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8706785d-9e46-4c00-95b0-7940ebcf3814 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:48:06.636884 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8706785d-9e46-4c00-95b0-7940ebcf3814 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:48:06.637012 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8706785d-9e46-4c00-95b0-7940ebcf3814 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:48:06.656855 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-8706785d-9e46-4c00-95b0-7940ebcf3814 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:48:06.658119 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-8706785d-9e46-4c00-95b0-7940ebcf3814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:48:06.727614 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1109/2204] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:48:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:48:06.735494 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f504947-0934-4a05-bfd1-5ecf3fd4c406 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:48:06.739077 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f504947-0934-4a05-bfd1-5ecf3fd4c406 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:48:06.739077 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f504947-0934-4a05-bfd1-5ecf3fd4c406 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:48:06.755472 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-6f504947-0934-4a05-bfd1-5ecf3fd4c406 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:48:06.756763 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-6f504947-0934-4a05-bfd1-5ecf3fd4c406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:48:06.765245 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1096/2205] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:48:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:48:06.769933 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8706785d-9e46-4c00-95b0-7940ebcf3814 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:48:06.780401 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8706785d-9e46-4c00-95b0-7940ebcf3814 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:48:06.785183 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500662bb-5a0c-4faf-a387-39428cc34daa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:48:06.785864 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500662bb-5a0c-4faf-a387-39428cc34daa admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:48:06.786156 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500662bb-5a0c-4faf-a387-39428cc34daa admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:48:06.786440 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500662bb-5a0c-4faf-a387-39428cc34daa admin admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:48:06.787205 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500662bb-5a0c-4faf-a387-39428cc34daa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:48:06.787524 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500662bb-5a0c-4faf-a387-39428cc34daa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:48:06.789222 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500662bb-5a0c-4faf-a387-39428cc34daa admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:48:06.802512 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1110/2206] 10.222.0.81 () {58 vars in 1152 bytes} [Wed Jul 20 22:48:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:48:06.807028 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f504947-0934-4a05-bfd1-5ecf3fd4c406 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:48:06.817246 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f504947-0934-4a05-bfd1-5ecf3fd4c406 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:48:06.821867 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96b7d5f-d1b0-4692-ac89-8c340f04323f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:48:06.822512 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96b7d5f-d1b0-4692-ac89-8c340f04323f admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:48:06.822884 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96b7d5f-d1b0-4692-ac89-8c340f04323f admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:48:06.823134 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96b7d5f-d1b0-4692-ac89-8c340f04323f admin admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:48:06.825686 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96b7d5f-d1b0-4692-ac89-8c340f04323f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:48:06.825797 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96b7d5f-d1b0-4692-ac89-8c340f04323f admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:48:06.922483 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1097/2207] 10.222.0.81 () {60 vars in 1152 bytes} [Wed Jul 20 22:48:06 2022] POST /identity/v3/projects => generated 371 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:48:06.923743 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500662bb-5a0c-4faf-a387-39428cc34daa admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:48:06.934635 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500662bb-5a0c-4faf-a387-39428cc34daa admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:48:06.992989 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb134e1-5801-4198-9f24-afe3f00bf883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:48:06.993682 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb134e1-5801-4198-9f24-afe3f00bf883 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:48:06.993970 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb134e1-5801-4198-9f24-afe3f00bf883 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:48:06.994250 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb134e1-5801-4198-9f24-afe3f00bf883 admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:48:06.994957 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb134e1-5801-4198-9f24-afe3f00bf883 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:48:06.997041 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb134e1-5801-4198-9f24-afe3f00bf883 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:48:07.106066 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1111/2208] 10.222.0.81 () {60 vars in 1143 bytes} [Wed Jul 20 22:48:06 2022] POST /identity/v3/users => generated 286 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 22:48:07.113090 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96b7d5f-d1b0-4692-ac89-8c340f04323f admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:48:07.123906 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96b7d5f-d1b0-4692-ac89-8c340f04323f admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:48:07.136321 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1301938c-4d69-4584-a918-928a05ca84ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:48:07.136321 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1301938c-4d69-4584-a918-928a05ca84ae admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:48:07.136321 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1301938c-4d69-4584-a918-928a05ca84ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:48:07.136321 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1301938c-4d69-4584-a918-928a05ca84ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:48:07.136689 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1301938c-4d69-4584-a918-928a05ca84ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:48:07.136689 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1301938c-4d69-4584-a918-928a05ca84ae admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:48:07.137278 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb134e1-5801-4198-9f24-afe3f00bf883 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:48:07.138993 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-0bb134e1-5801-4198-9f24-afe3f00bf883 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:48:07.151152 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb134e1-5801-4198-9f24-afe3f00bf883 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:48:07.156307 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126bac76-5303-44c0-bff3-b5511c4928d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:48:07.157028 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126bac76-5303-44c0-bff3-b5511c4928d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:48:07.157315 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126bac76-5303-44c0-bff3-b5511c4928d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:48:07.157736 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126bac76-5303-44c0-bff3-b5511c4928d2 admin admin] PATH_INFO: `/v3/users/2b09f9a730724f4ebc3415be7a510fac` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:48:07.159472 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126bac76-5303-44c0-bff3-b5511c4928d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b09f9a730724f4ebc3415be7a510fac)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:48:07.161427 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126bac76-5303-44c0-bff3-b5511c4928d2 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:48:07.176640 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1098/2209] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:48:07 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:48:07.187049 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1301938c-4d69-4584-a918-928a05ca84ae admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:48:07.202857 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1301938c-4d69-4584-a918-928a05ca84ae admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:48:07.209058 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca00f989-dae1-42c2-a4e4-af97a576a53f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:48:07.209058 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca00f989-dae1-42c2-a4e4-af97a576a53f admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:48:07.209058 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca00f989-dae1-42c2-a4e4-af97a576a53f admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:48:07.209058 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca00f989-dae1-42c2-a4e4-af97a576a53f admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:48:07.209681 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca00f989-dae1-42c2-a4e4-af97a576a53f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:48:07.211306 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca00f989-dae1-42c2-a4e4-af97a576a53f admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:48:07.222596 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1099/2210] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:48:07 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:48:07.225967 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca00f989-dae1-42c2-a4e4-af97a576a53f admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:48:07.235793 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca00f989-dae1-42c2-a4e4-af97a576a53f admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:48:07.241205 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fdac6e-3f32-42c6-9229-7baec600f8f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:48:07.242102 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fdac6e-3f32-42c6-9229-7baec600f8f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:48:07.242423 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fdac6e-3f32-42c6-9229-7baec600f8f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:48:07.242710 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fdac6e-3f32-42c6-9229-7baec600f8f2 admin admin] PATH_INFO: `/v3/projects/fffefe641e7b462aab15453e9611f319/users/fdb4cffdbe9b4b139cceb6d8ee867b8d/roles/559942c4de1744169f8b5e0404ecde6b` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:48:07.257270 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fdac6e-3f32-42c6-9229-7baec600f8f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fffefe641e7b462aab15453e9611f319, user_id=fdb4cffdbe9b4b139cceb6d8ee867b8d, role_id=559942c4de1744169f8b5e0404ecde6b)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:48:07.259945 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fdac6e-3f32-42c6-9229-7baec600f8f2 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:48:07.341408 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1100/2211] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:48:07 2022] PUT /identity/v3/projects/fffefe641e7b462aab15453e9611f319/users/fdb4cffdbe9b4b139cceb6d8ee867b8d/roles/559942c4de1744169f8b5e0404ecde6b => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 22:48:07.353138 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19438c55-0719-443a-ba4c-4d053dd8e6f1 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:48:07.353435 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19438c55-0719-443a-ba4c-4d053dd8e6f1 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:48:07.353718 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19438c55-0719-443a-ba4c-4d053dd8e6f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:48:07.372317 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:48:07.388990 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-19438c55-0719-443a-ba4c-4d053dd8e6f1 None None] MFA Rules not processed for user `fdb4cffdbe9b4b139cceb6d8ee867b8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:48:07.390283 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-19438c55-0719-443a-ba4c-4d053dd8e6f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:48:07.393664 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-126bac76-5303-44c0-bff3-b5511c4928d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b09f9a730724f4ebc3415be7a510fac'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:48:07.400240 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-126bac76-5303-44c0-bff3-b5511c4928d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b09f9a730724f4ebc3415be7a510fac'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:48:07.401350 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-126bac76-5303-44c0-bff3-b5511c4928d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b09f9a730724f4ebc3415be7a510fac'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:48:07.413326 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-126bac76-5303-44c0-bff3-b5511c4928d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b09f9a730724f4ebc3415be7a510fac'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:48:07.467405 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1101/2212] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:48:07 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 22:48:07.475955 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19438c55-0719-443a-ba4c-4d053dd8e6f1 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:48:07.477362 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-19438c55-0719-443a-ba4c-4d053dd8e6f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:48:07.505495 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-126bac76-5303-44c0-bff3-b5511c4928d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b09f9a730724f4ebc3415be7a510fac'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:48:07.531535 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1112/2213] 10.222.0.81 () {58 vars in 1223 bytes} [Wed Jul 20 22:48:07 2022] DELETE /identity/v3/users/2b09f9a730724f4ebc3415be7a510fac => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 22:48:07.536800 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc44bcb3-5197-4952-8494-a1c5096ca570 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:48:07.537102 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc44bcb3-5197-4952-8494-a1c5096ca570 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:48:07.537474 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc44bcb3-5197-4952-8494-a1c5096ca570 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:48:07.551161 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19438c55-0719-443a-ba4c-4d053dd8e6f1 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:48:07.567674 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-dc44bcb3-5197-4952-8494-a1c5096ca570 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:48:07.569035 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-dc44bcb3-5197-4952-8494-a1c5096ca570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:48:07.590840 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:DELETE=1,keystone:SELECT=97,keystone:INSERT=2 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:48:07.645262 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2fc4ac-9d85-4dce-9cea-3081e9b03f63 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c44640ea0014426a9523bb1c457778b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:48:07.646420 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2fc4ac-9d85-4dce-9cea-3081e9b03f63 service glance] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:48:07.646937 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2fc4ac-9d85-4dce-9cea-3081e9b03f63 service glance] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:48:07.647436 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2fc4ac-9d85-4dce-9cea-3081e9b03f63 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:48:07.671344 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1113/2214] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:48:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:48:07.693389 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc44bcb3-5197-4952-8494-a1c5096ca570 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:48:07.695717 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-dc44bcb3-5197-4952-8494-a1c5096ca570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:48:07.741691 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2fc4ac-9d85-4dce-9cea-3081e9b03f63 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:48:07.743502 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2fc4ac-9d85-4dce-9cea-3081e9b03f63 service glance] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:48:07.755045 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1102/2215] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:48:07 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 22:48:07.785405 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc44bcb3-5197-4952-8494-a1c5096ca570 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:48:07.798603 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca817798-a075-4ee3-a3a8-57c764bd4d1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:48:07.799495 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca817798-a075-4ee3-a3a8-57c764bd4d1a service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:48:07.800003 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca817798-a075-4ee3-a3a8-57c764bd4d1a service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:48:07.800429 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca817798-a075-4ee3-a3a8-57c764bd4d1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:48:07.812232 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca817798-a075-4ee3-a3a8-57c764bd4d1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:48:07.814069 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca817798-a075-4ee3-a3a8-57c764bd4d1a service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:48:07.827972 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1114/2216] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:48:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 22:48:08.307792 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2fc4ac-9d85-4dce-9cea-3081e9b03f63 service glance] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:48:08.309200 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-be2fc4ac-9d85-4dce-9cea-3081e9b03f63 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:48:08.337302 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2fc4ac-9d85-4dce-9cea-3081e9b03f63 service glance] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:48:08.341792 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0795159f-cbd5-4115-b0f7-f81a5bb5c051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:48:08.342961 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0795159f-cbd5-4115-b0f7-f81a5bb5c051 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:48:08.342961 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0795159f-cbd5-4115-b0f7-f81a5bb5c051 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:48:08.343406 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0795159f-cbd5-4115-b0f7-f81a5bb5c051 admin admin] PATH_INFO: `/v3/projects/19c2bac554864487b87772c8d8663a3d` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:48:08.344617 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0795159f-cbd5-4115-b0f7-f81a5bb5c051 admin admin] RBAC: Authorizing `identity:delete_project(project_id=19c2bac554864487b87772c8d8663a3d)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:48:08.346540 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0795159f-cbd5-4115-b0f7-f81a5bb5c051 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:48:08.497573 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-0795159f-cbd5-4115-b0f7-f81a5bb5c051 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19c2bac554864487b87772c8d8663a3d', 'request_id': 'req-0795159f-cbd5-4115-b0f7-f81a5bb5c051'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:48:08.591429 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-0795159f-cbd5-4115-b0f7-f81a5bb5c051 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19c2bac554864487b87772c8d8663a3d', 'request_id': 'req-0795159f-cbd5-4115-b0f7-f81a5bb5c051'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:48:08.600382 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-0795159f-cbd5-4115-b0f7-f81a5bb5c051 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19c2bac554864487b87772c8d8663a3d', 'request_id': 'req-0795159f-cbd5-4115-b0f7-f81a5bb5c051'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:48:08.617619 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-0795159f-cbd5-4115-b0f7-f81a5bb5c051 admin admin] The token cache is being invalidate because project 19c2bac554864487b87772c8d8663a3d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:48:08.617917 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-0795159f-cbd5-4115-b0f7-f81a5bb5c051 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:48:08.618873 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-0795159f-cbd5-4115-b0f7-f81a5bb5c051 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:48:08.620563 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1103/2217] 10.222.0.81 () {58 vars in 1232 bytes} [Wed Jul 20 22:48:08 2022] DELETE /identity/v3/projects/19c2bac554864487b87772c8d8663a3d => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:48:08.625545 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35feaced-e7ff-4f51-85ca-7c48c911bb3f None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:48:08.630263 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35feaced-e7ff-4f51-85ca-7c48c911bb3f None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:48:08.630263 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35feaced-e7ff-4f51-85ca-7c48c911bb3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:48:08.644620 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-35feaced-e7ff-4f51-85ca-7c48c911bb3f None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:48:08.646010 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-35feaced-e7ff-4f51-85ca-7c48c911bb3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:48:08.697584 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:48:08.710947 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1115/2218] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:48:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:48:08.720208 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65b00d0-cbff-4a7c-9d7d-b34b4f716730 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:48:08.720543 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65b00d0-cbff-4a7c-9d7d-b34b4f716730 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:48:08.720906 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65b00d0-cbff-4a7c-9d7d-b34b4f716730 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:48:08.755620 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-d65b00d0-cbff-4a7c-9d7d-b34b4f716730 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:48:08.757121 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-d65b00d0-cbff-4a7c-9d7d-b34b4f716730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:48:08.771334 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1104/2219] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:48:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:48:08.776137 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35feaced-e7ff-4f51-85ca-7c48c911bb3f None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:48:08.805497 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35feaced-e7ff-4f51-85ca-7c48c911bb3f None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:48:08.821374 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4dc049-7953-48ee-9965-1dc44e6b83b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:48:08.822352 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4dc049-7953-48ee-9965-1dc44e6b83b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:48:08.833964 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4dc049-7953-48ee-9965-1dc44e6b83b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:48:08.833964 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4dc049-7953-48ee-9965-1dc44e6b83b5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:48:08.833964 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4dc049-7953-48ee-9965-1dc44e6b83b5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:48:08.833964 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4dc049-7953-48ee-9965-1dc44e6b83b5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:48:08.833964 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4dc049-7953-48ee-9965-1dc44e6b83b5 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:48:08.844233 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1116/2220] 10.222.0.81 () {58 vars in 1152 bytes} [Wed Jul 20 22:48:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:48:08.849051 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65b00d0-cbff-4a7c-9d7d-b34b4f716730 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:48:08.860236 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65b00d0-cbff-4a7c-9d7d-b34b4f716730 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:48:08.867079 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dc94a0-43dc-4604-91a6-3ee12c9012aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:48:08.867910 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dc94a0-43dc-4604-91a6-3ee12c9012aa admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:48:08.868372 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dc94a0-43dc-4604-91a6-3ee12c9012aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:48:08.868875 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dc94a0-43dc-4604-91a6-3ee12c9012aa admin admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:48:08.869774 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dc94a0-43dc-4604-91a6-3ee12c9012aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:48:08.873104 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dc94a0-43dc-4604-91a6-3ee12c9012aa admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:48:08.957894 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1105/2221] 10.222.0.81 () {60 vars in 1152 bytes} [Wed Jul 20 22:48:08 2022] POST /identity/v3/projects => generated 365 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:48:08.961868 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4dc049-7953-48ee-9965-1dc44e6b83b5 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:48:08.973355 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4dc049-7953-48ee-9965-1dc44e6b83b5 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:48:09.044253 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0708b875-a893-42d7-8d70-5f120a622c61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:48:09.045268 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0708b875-a893-42d7-8d70-5f120a622c61 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:48:09.046060 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0708b875-a893-42d7-8d70-5f120a622c61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:48:09.046060 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0708b875-a893-42d7-8d70-5f120a622c61 admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:48:09.047180 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0708b875-a893-42d7-8d70-5f120a622c61 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:48:09.049345 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0708b875-a893-42d7-8d70-5f120a622c61 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:48:09.163555 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1117/2222] 10.222.0.81 () {60 vars in 1143 bytes} [Wed Jul 20 22:48:08 2022] POST /identity/v3/users => generated 283 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:48:09.168055 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dc94a0-43dc-4604-91a6-3ee12c9012aa admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:48:09.178168 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dc94a0-43dc-4604-91a6-3ee12c9012aa admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:48:09.182714 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9091579-196e-43a6-bd92-860e1b4531b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:48:09.183424 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9091579-196e-43a6-bd92-860e1b4531b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:48:09.183714 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9091579-196e-43a6-bd92-860e1b4531b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:48:09.184026 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9091579-196e-43a6-bd92-860e1b4531b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:48:09.184807 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9091579-196e-43a6-bd92-860e1b4531b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:48:09.186444 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9091579-196e-43a6-bd92-860e1b4531b3 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:48:09.199143 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1106/2223] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:48:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:48:09.201913 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0708b875-a893-42d7-8d70-5f120a622c61 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:48:09.214637 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0708b875-a893-42d7-8d70-5f120a622c61 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:48:09.219716 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb6bce7-302d-42c0-b029-d1bd036228f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:48:09.220388 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb6bce7-302d-42c0-b029-d1bd036228f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:48:09.222127 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb6bce7-302d-42c0-b029-d1bd036228f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:48:09.222127 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb6bce7-302d-42c0-b029-d1bd036228f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:48:09.222318 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb6bce7-302d-42c0-b029-d1bd036228f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:48:09.224005 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb6bce7-302d-42c0-b029-d1bd036228f9 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:48:09.238149 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1118/2224] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:48:09 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:48:09.239629 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9091579-196e-43a6-bd92-860e1b4531b3 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:48:09.253830 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9091579-196e-43a6-bd92-860e1b4531b3 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:48:09.258776 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f148fd0-668e-4bb1-9605-3ff2add30de7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:48:09.259857 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f148fd0-668e-4bb1-9605-3ff2add30de7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:48:09.260139 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f148fd0-668e-4bb1-9605-3ff2add30de7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:48:09.260757 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f148fd0-668e-4bb1-9605-3ff2add30de7 admin admin] PATH_INFO: `/v3/projects/80a25ac1672c48cb897167787b0ffbcf/users/572ad0393df74b03aabd04866dfcd05d/roles/559942c4de1744169f8b5e0404ecde6b` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:48:09.284613 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f148fd0-668e-4bb1-9605-3ff2add30de7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80a25ac1672c48cb897167787b0ffbcf, user_id=572ad0393df74b03aabd04866dfcd05d, role_id=559942c4de1744169f8b5e0404ecde6b)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:48:09.287846 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f148fd0-668e-4bb1-9605-3ff2add30de7 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:48:09.290549 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:48:09.359104 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1107/2225] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:48:09 2022] PUT /identity/v3/projects/80a25ac1672c48cb897167787b0ffbcf/users/572ad0393df74b03aabd04866dfcd05d/roles/559942c4de1744169f8b5e0404ecde6b => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:48:09.369259 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244429f7-9ddd-4351-ab4b-5e0715f8ccd9 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:48:09.369553 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244429f7-9ddd-4351-ab4b-5e0715f8ccd9 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:48:09.369842 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244429f7-9ddd-4351-ab4b-5e0715f8ccd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:48:09.413148 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-244429f7-9ddd-4351-ab4b-5e0715f8ccd9 None None] MFA Rules not processed for user `572ad0393df74b03aabd04866dfcd05d`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:48:09.415465 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-244429f7-9ddd-4351-ab4b-5e0715f8ccd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:48:09.530773 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1119/2226] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:48:09 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 22:48:09.535345 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f148fd0-668e-4bb1-9605-3ff2add30de7 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:48:09.610109 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f148fd0-668e-4bb1-9605-3ff2add30de7 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:48:09.615798 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1740910-81a9-4294-a09c-24d212e12b80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:48:09.616694 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1740910-81a9-4294-a09c-24d212e12b80 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:48:09.617060 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1740910-81a9-4294-a09c-24d212e12b80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:48:09.617411 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1740910-81a9-4294-a09c-24d212e12b80 admin admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:48:09.618199 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1740910-81a9-4294-a09c-24d212e12b80 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:48:09.620290 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1740910-81a9-4294-a09c-24d212e12b80 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:48:09.760043 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1108/2227] 10.222.0.81 () {60 vars in 1152 bytes} [Wed Jul 20 22:48:09 2022] POST /identity/v3/projects => generated 365 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 22:48:09.761069 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244429f7-9ddd-4351-ab4b-5e0715f8ccd9 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:48:09.773268 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244429f7-9ddd-4351-ab4b-5e0715f8ccd9 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:48:09.866417 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:48:09.879204 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8320e2-0e59-46f3-8228-39b998a1bbc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:48:09.881660 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8320e2-0e59-46f3-8228-39b998a1bbc7 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:48:09.881660 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8320e2-0e59-46f3-8228-39b998a1bbc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:48:09.881660 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8320e2-0e59-46f3-8228-39b998a1bbc7 admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:48:09.881660 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8320e2-0e59-46f3-8228-39b998a1bbc7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:48:09.883229 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8320e2-0e59-46f3-8228-39b998a1bbc7 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:48:09.992701 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1120/2228] 10.222.0.81 () {60 vars in 1143 bytes} [Wed Jul 20 22:48:09 2022] POST /identity/v3/users => generated 289 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 22:48:09.996928 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1740910-81a9-4294-a09c-24d212e12b80 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:48:10.008386 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1740910-81a9-4294-a09c-24d212e12b80 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:48:10.013218 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430ac0f8-124c-483e-b0fc-eebaa1a6dde6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:48:10.018135 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430ac0f8-124c-483e-b0fc-eebaa1a6dde6 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:48:10.018135 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430ac0f8-124c-483e-b0fc-eebaa1a6dde6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:48:10.018135 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430ac0f8-124c-483e-b0fc-eebaa1a6dde6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:48:10.018135 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430ac0f8-124c-483e-b0fc-eebaa1a6dde6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:48:10.018135 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430ac0f8-124c-483e-b0fc-eebaa1a6dde6 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:48:10.030083 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1109/2229] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:48:09 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 22:48:10.034831 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8320e2-0e59-46f3-8228-39b998a1bbc7 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:48:10.044664 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8320e2-0e59-46f3-8228-39b998a1bbc7 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:48:10.049273 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4e448c-16db-4e06-8a31-3c15027e4dac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:48:10.050767 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4e448c-16db-4e06-8a31-3c15027e4dac admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:48:10.051074 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4e448c-16db-4e06-8a31-3c15027e4dac admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:48:10.051353 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4e448c-16db-4e06-8a31-3c15027e4dac admin admin] PATH_INFO: `/v3/projects/843162eb36594e958b8037a47194a396/users/40e58d416f924274829d8d59c67ee9f1/roles/d89b6c92d35a401a8517f1d47f6be843` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:48:10.066216 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4e448c-16db-4e06-8a31-3c15027e4dac admin admin] RBAC: Authorizing `identity:create_grant(project_id=843162eb36594e958b8037a47194a396, user_id=40e58d416f924274829d8d59c67ee9f1, role_id=d89b6c92d35a401a8517f1d47f6be843)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:48:10.068892 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4e448c-16db-4e06-8a31-3c15027e4dac admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:48:10.134321 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1121/2230] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:48:10 2022] PUT /identity/v3/projects/843162eb36594e958b8037a47194a396/users/40e58d416f924274829d8d59c67ee9f1/roles/d89b6c92d35a401a8517f1d47f6be843 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:48:10.139047 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430ac0f8-124c-483e-b0fc-eebaa1a6dde6 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:48:10.148971 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430ac0f8-124c-483e-b0fc-eebaa1a6dde6 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:48:10.214359 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8fe564-8c3c-42e0-8231-7933290c3ac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:48:10.215073 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8fe564-8c3c-42e0-8231-7933290c3ac1 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:48:10.215366 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8fe564-8c3c-42e0-8231-7933290c3ac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:48:10.215650 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8fe564-8c3c-42e0-8231-7933290c3ac1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:48:10.216389 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8fe564-8c3c-42e0-8231-7933290c3ac1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:48:10.218101 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8fe564-8c3c-42e0-8231-7933290c3ac1 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:48:10.229396 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1110/2231] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:48:10 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:48:10.233716 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4e448c-16db-4e06-8a31-3c15027e4dac admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:48:10.244867 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4e448c-16db-4e06-8a31-3c15027e4dac admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:48:10.249564 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a3f3a3-73d4-4998-8b41-3b07c6e516b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:48:10.250475 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a3f3a3-73d4-4998-8b41-3b07c6e516b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:48:10.250761 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a3f3a3-73d4-4998-8b41-3b07c6e516b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:48:10.251067 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a3f3a3-73d4-4998-8b41-3b07c6e516b4 admin admin] PATH_INFO: `/v3/projects/843162eb36594e958b8037a47194a396/users/40e58d416f924274829d8d59c67ee9f1/roles/d89b6c92d35a401a8517f1d47f6be843` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:48:10.253097 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a3f3a3-73d4-4998-8b41-3b07c6e516b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=843162eb36594e958b8037a47194a396, user_id=40e58d416f924274829d8d59c67ee9f1, role_id=d89b6c92d35a401a8517f1d47f6be843)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:48:10.255899 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a3f3a3-73d4-4998-8b41-3b07c6e516b4 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:48:10.266504 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1122/2232] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:48:10 2022] PUT /identity/v3/projects/843162eb36594e958b8037a47194a396/users/40e58d416f924274829d8d59c67ee9f1/roles/d89b6c92d35a401a8517f1d47f6be843 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:48:10.276959 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4754f3ff-fa2b-4ce8-9ae1-3c1de7a81630 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:48:10.277251 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4754f3ff-fa2b-4ce8-9ae1-3c1de7a81630 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:48:10.277533 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4754f3ff-fa2b-4ce8-9ae1-3c1de7a81630 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:48:10.306686 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-4754f3ff-fa2b-4ce8-9ae1-3c1de7a81630 None None] MFA Rules not processed for user `40e58d416f924274829d8d59c67ee9f1`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:48:10.307939 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-4754f3ff-fa2b-4ce8-9ae1-3c1de7a81630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:48:10.368070 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:48:10.387001 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1111/2233] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:48:10 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:48:10.398710 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a3f3a3-73d4-4998-8b41-3b07c6e516b4 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:48:10.401140 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-f0a3f3a3-73d4-4998-8b41-3b07c6e516b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:48:10.464825 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a3f3a3-73d4-4998-8b41-3b07c6e516b4 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:48:10.469042 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e869ca63-6d63-41f4-b16f-78e990b1e771 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecf4db289fba4e4a806198c62d31f7c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:48:10.469788 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e869ca63-6d63-41f4-b16f-78e990b1e771 service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:48:10.469788 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e869ca63-6d63-41f4-b16f-78e990b1e771 service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:48:10.470037 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e869ca63-6d63-41f4-b16f-78e990b1e771 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:48:10.480884 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e869ca63-6d63-41f4-b16f-78e990b1e771 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:48:10.482420 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e869ca63-6d63-41f4-b16f-78e990b1e771 service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:48:10.493045 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1123/2234] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:48:10 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:48:11.459481 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4754f3ff-fa2b-4ce8-9ae1-3c1de7a81630 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:48:11.529565 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4754f3ff-fa2b-4ce8-9ae1-3c1de7a81630 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:48:11.535347 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec8afd9-92d0-4a4b-a48b-8257ac7d8bcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:48:11.536213 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec8afd9-92d0-4a4b-a48b-8257ac7d8bcc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:48:11.536502 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec8afd9-92d0-4a4b-a48b-8257ac7d8bcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:48:11.536788 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec8afd9-92d0-4a4b-a48b-8257ac7d8bcc admin admin] PATH_INFO: `/v3/users/572ad0393df74b03aabd04866dfcd05d` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:48:11.538069 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec8afd9-92d0-4a4b-a48b-8257ac7d8bcc admin admin] RBAC: Authorizing `identity:delete_user(user_id=572ad0393df74b03aabd04866dfcd05d)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:48:11.540118 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec8afd9-92d0-4a4b-a48b-8257ac7d8bcc admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:48:11.725618 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-dec8afd9-92d0-4a4b-a48b-8257ac7d8bcc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '572ad0393df74b03aabd04866dfcd05d'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:48:11.726186 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-dec8afd9-92d0-4a4b-a48b-8257ac7d8bcc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '572ad0393df74b03aabd04866dfcd05d'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:48:11.726880 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-dec8afd9-92d0-4a4b-a48b-8257ac7d8bcc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '572ad0393df74b03aabd04866dfcd05d'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:48:11.734455 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-dec8afd9-92d0-4a4b-a48b-8257ac7d8bcc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '572ad0393df74b03aabd04866dfcd05d'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:48:11.814100 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-dec8afd9-92d0-4a4b-a48b-8257ac7d8bcc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '572ad0393df74b03aabd04866dfcd05d'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:48:11.836409 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1112/2235] 10.222.0.81 () {58 vars in 1223 bytes} [Wed Jul 20 22:48:11 2022] DELETE /identity/v3/users/572ad0393df74b03aabd04866dfcd05d => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:48:11.840933 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad32ae5-5492-4486-99c7-93b614cf0c21 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:48:11.841437 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad32ae5-5492-4486-99c7-93b614cf0c21 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:48:11.841535 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad32ae5-5492-4486-99c7-93b614cf0c21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:48:11.863255 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-aad32ae5-5492-4486-99c7-93b614cf0c21 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:48:11.864684 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-aad32ae5-5492-4486-99c7-93b614cf0c21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:48:11.942318 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1124/2236] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:48:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:48:11.952285 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec8afd9-92d0-4a4b-a48b-8257ac7d8bcc admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:48:11.954219 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-dec8afd9-92d0-4a4b-a48b-8257ac7d8bcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:48:12.013622 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec8afd9-92d0-4a4b-a48b-8257ac7d8bcc admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:48:12.017184 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be659bae-96a8-4761-8c65-aeb044326312 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:48:12.017870 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be659bae-96a8-4761-8c65-aeb044326312 service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:48:12.018692 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be659bae-96a8-4761-8c65-aeb044326312 service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:48:12.018973 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be659bae-96a8-4761-8c65-aeb044326312 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:48:12.028230 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be659bae-96a8-4761-8c65-aeb044326312 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:48:12.029915 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be659bae-96a8-4761-8c65-aeb044326312 service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:48:12.039913 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1113/2237] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:48:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:48:12.445070 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad32ae5-5492-4486-99c7-93b614cf0c21 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:48:12.445070 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-aad32ae5-5492-4486-99c7-93b614cf0c21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:48:12.459071 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad32ae5-5492-4486-99c7-93b614cf0c21 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:48:12.463806 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaf2908-0ff7-4e56-a1c7-0eb16c03742e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:48:12.464793 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaf2908-0ff7-4e56-a1c7-0eb16c03742e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:48:12.465080 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaf2908-0ff7-4e56-a1c7-0eb16c03742e admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:48:12.465365 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaf2908-0ff7-4e56-a1c7-0eb16c03742e admin admin] PATH_INFO: `/v3/projects/80a25ac1672c48cb897167787b0ffbcf` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:48:12.466891 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaf2908-0ff7-4e56-a1c7-0eb16c03742e admin admin] RBAC: Authorizing `identity:delete_project(project_id=80a25ac1672c48cb897167787b0ffbcf)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:48:12.468880 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaf2908-0ff7-4e56-a1c7-0eb16c03742e admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:48:12.574918 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:48:12.582197 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-7aaf2908-0ff7-4e56-a1c7-0eb16c03742e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80a25ac1672c48cb897167787b0ffbcf', 'request_id': 'req-7aaf2908-0ff7-4e56-a1c7-0eb16c03742e'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:48:12.588485 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-7aaf2908-0ff7-4e56-a1c7-0eb16c03742e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80a25ac1672c48cb897167787b0ffbcf', 'request_id': 'req-7aaf2908-0ff7-4e56-a1c7-0eb16c03742e'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:48:12.682652 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-7aaf2908-0ff7-4e56-a1c7-0eb16c03742e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80a25ac1672c48cb897167787b0ffbcf', 'request_id': 'req-7aaf2908-0ff7-4e56-a1c7-0eb16c03742e'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:48:12.694551 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-7aaf2908-0ff7-4e56-a1c7-0eb16c03742e admin admin] The token cache is being invalidate because project 80a25ac1672c48cb897167787b0ffbcf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:48:12.694844 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-7aaf2908-0ff7-4e56-a1c7-0eb16c03742e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:48:12.695535 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-7aaf2908-0ff7-4e56-a1c7-0eb16c03742e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:48:12.697182 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1125/2238] 10.222.0.81 () {58 vars in 1232 bytes} [Wed Jul 20 22:48:12 2022] DELETE /identity/v3/projects/80a25ac1672c48cb897167787b0ffbcf => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:48:12.701208 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be659bae-96a8-4761-8c65-aeb044326312 service neutron] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:48:12.711438 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be659bae-96a8-4761-8c65-aeb044326312 service neutron] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:48:12.719349 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:48:12.771251 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f1d1b7-7ad5-4310-9042-3991d0b3959e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:48:12.771251 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f1d1b7-7ad5-4310-9042-3991d0b3959e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:48:12.771590 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f1d1b7-7ad5-4310-9042-3991d0b3959e admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:48:12.771712 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f1d1b7-7ad5-4310-9042-3991d0b3959e admin admin] PATH_INFO: `/v3/users/40e58d416f924274829d8d59c67ee9f1` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:48:12.773372 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f1d1b7-7ad5-4310-9042-3991d0b3959e admin admin] RBAC: Authorizing `identity:delete_user(user_id=40e58d416f924274829d8d59c67ee9f1)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:48:12.775490 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f1d1b7-7ad5-4310-9042-3991d0b3959e admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:48:12.969631 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-e3f1d1b7-7ad5-4310-9042-3991d0b3959e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40e58d416f924274829d8d59c67ee9f1'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:48:12.973291 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-e3f1d1b7-7ad5-4310-9042-3991d0b3959e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40e58d416f924274829d8d59c67ee9f1'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:48:12.973960 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-e3f1d1b7-7ad5-4310-9042-3991d0b3959e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40e58d416f924274829d8d59c67ee9f1'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:48:12.984201 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-e3f1d1b7-7ad5-4310-9042-3991d0b3959e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40e58d416f924274829d8d59c67ee9f1'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:48:13.054527 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-e3f1d1b7-7ad5-4310-9042-3991d0b3959e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40e58d416f924274829d8d59c67ee9f1'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:48:13.073695 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1114/2239] 10.222.0.81 () {58 vars in 1223 bytes} [Wed Jul 20 22:48:12 2022] DELETE /identity/v3/users/40e58d416f924274829d8d59c67ee9f1 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:48:13.542450 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaf2908-0ff7-4e56-a1c7-0eb16c03742e admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:48:13.545193 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-7aaf2908-0ff7-4e56-a1c7-0eb16c03742e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:48:13.560186 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaf2908-0ff7-4e56-a1c7-0eb16c03742e admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:48:13.617625 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1299b142-3aec-4866-8c8a-1f2b8ccee1fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:48:13.618647 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1299b142-3aec-4866-8c8a-1f2b8ccee1fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:48:13.618939 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1299b142-3aec-4866-8c8a-1f2b8ccee1fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:48:13.619249 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1299b142-3aec-4866-8c8a-1f2b8ccee1fa admin admin] PATH_INFO: `/v3/projects/843162eb36594e958b8037a47194a396` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:48:13.620446 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1299b142-3aec-4866-8c8a-1f2b8ccee1fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=843162eb36594e958b8037a47194a396)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:48:13.622394 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1299b142-3aec-4866-8c8a-1f2b8ccee1fa admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:48:13.741142 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-1299b142-3aec-4866-8c8a-1f2b8ccee1fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '843162eb36594e958b8037a47194a396', 'request_id': 'req-1299b142-3aec-4866-8c8a-1f2b8ccee1fa'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:48:13.747354 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-1299b142-3aec-4866-8c8a-1f2b8ccee1fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '843162eb36594e958b8037a47194a396', 'request_id': 'req-1299b142-3aec-4866-8c8a-1f2b8ccee1fa'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:48:13.813535 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-1299b142-3aec-4866-8c8a-1f2b8ccee1fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '843162eb36594e958b8037a47194a396', 'request_id': 'req-1299b142-3aec-4866-8c8a-1f2b8ccee1fa'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:48:13.824960 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-1299b142-3aec-4866-8c8a-1f2b8ccee1fa admin admin] The token cache is being invalidate because project 843162eb36594e958b8037a47194a396 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:48:13.825255 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-1299b142-3aec-4866-8c8a-1f2b8ccee1fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:48:13.825885 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-1299b142-3aec-4866-8c8a-1f2b8ccee1fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:48:13.827493 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1126/2240] 10.222.0.81 () {58 vars in 1232 bytes} [Wed Jul 20 22:48:13 2022] DELETE /identity/v3/projects/843162eb36594e958b8037a47194a396 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:48:13.833281 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7548c9-882a-4c6f-becb-074200df3b58 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:48:13.833842 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7548c9-882a-4c6f-becb-074200df3b58 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:48:13.840493 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7548c9-882a-4c6f-becb-074200df3b58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:48:13.852158 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-ec7548c9-882a-4c6f-becb-074200df3b58 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:48:13.853362 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-ec7548c9-882a-4c6f-becb-074200df3b58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:48:13.913736 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1115/2241] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:48:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:48:13.921562 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c3da93-13bc-4123-87a1-c67fc19ca632 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:48:13.921858 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c3da93-13bc-4123-87a1-c67fc19ca632 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:48:13.922146 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c3da93-13bc-4123-87a1-c67fc19ca632 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:48:13.941103 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-f0c3da93-13bc-4123-87a1-c67fc19ca632 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:48:13.942315 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-f0c3da93-13bc-4123-87a1-c67fc19ca632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:48:13.950214 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1127/2242] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:48:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:48:13.954665 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7548c9-882a-4c6f-becb-074200df3b58 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:48:13.965093 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7548c9-882a-4c6f-becb-074200df3b58 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:48:13.969757 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edaa973-ba0c-441f-afd0-d26dfbf177d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:48:13.970417 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edaa973-ba0c-441f-afd0-d26dfbf177d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:48:13.970710 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edaa973-ba0c-441f-afd0-d26dfbf177d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:48:13.970996 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edaa973-ba0c-441f-afd0-d26dfbf177d3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:48:13.971752 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edaa973-ba0c-441f-afd0-d26dfbf177d3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:48:13.972065 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edaa973-ba0c-441f-afd0-d26dfbf177d3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:48:13.973668 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edaa973-ba0c-441f-afd0-d26dfbf177d3 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:48:13.985108 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1116/2243] 10.222.0.81 () {58 vars in 1152 bytes} [Wed Jul 20 22:48:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:48:13.989965 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c3da93-13bc-4123-87a1-c67fc19ca632 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:48:13.999805 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c3da93-13bc-4123-87a1-c67fc19ca632 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:48:14.006263 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613e649e-ce2a-4cf4-8f6d-bd7f26eb3d38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:48:14.006936 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613e649e-ce2a-4cf4-8f6d-bd7f26eb3d38 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:48:14.007273 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613e649e-ce2a-4cf4-8f6d-bd7f26eb3d38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:48:14.007572 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613e649e-ce2a-4cf4-8f6d-bd7f26eb3d38 admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:48:14.008267 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613e649e-ce2a-4cf4-8f6d-bd7f26eb3d38 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:48:14.010183 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613e649e-ce2a-4cf4-8f6d-bd7f26eb3d38 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:48:14.121802 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1128/2244] 10.222.0.81 () {60 vars in 1152 bytes} [Wed Jul 20 22:48:13 2022] POST /identity/v3/projects => generated 365 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:48:14.123030 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edaa973-ba0c-441f-afd0-d26dfbf177d3 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:48:14.132915 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edaa973-ba0c-441f-afd0-d26dfbf177d3 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:48:14.150935 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:48:14.199517 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328c4f17-7119-4933-ae46-61c203978c34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:48:14.200248 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328c4f17-7119-4933-ae46-61c203978c34 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:48:14.200529 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328c4f17-7119-4933-ae46-61c203978c34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:48:14.200757 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328c4f17-7119-4933-ae46-61c203978c34 admin admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:48:14.201447 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328c4f17-7119-4933-ae46-61c203978c34 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:48:14.203405 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328c4f17-7119-4933-ae46-61c203978c34 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:48:14.306536 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1117/2245] 10.222.0.81 () {60 vars in 1143 bytes} [Wed Jul 20 22:48:14 2022] POST /identity/v3/users => generated 283 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:48:14.310746 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613e649e-ce2a-4cf4-8f6d-bd7f26eb3d38 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:48:14.321451 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613e649e-ce2a-4cf4-8f6d-bd7f26eb3d38 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:48:14.326053 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfafc8ca-8985-4ff2-91e6-9663da5744a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:48:14.326725 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfafc8ca-8985-4ff2-91e6-9663da5744a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:48:14.327039 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfafc8ca-8985-4ff2-91e6-9663da5744a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:48:14.332942 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfafc8ca-8985-4ff2-91e6-9663da5744a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:48:14.332942 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfafc8ca-8985-4ff2-91e6-9663da5744a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:48:14.332942 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfafc8ca-8985-4ff2-91e6-9663da5744a6 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:48:14.340655 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1129/2246] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:48:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:48:14.344739 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328c4f17-7119-4933-ae46-61c203978c34 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:48:14.355234 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328c4f17-7119-4933-ae46-61c203978c34 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:48:14.359748 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981e4426-cb4c-4c8f-a729-df5aacbbcbea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:48:14.360397 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981e4426-cb4c-4c8f-a729-df5aacbbcbea admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:48:14.360680 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981e4426-cb4c-4c8f-a729-df5aacbbcbea admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:48:14.360957 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981e4426-cb4c-4c8f-a729-df5aacbbcbea admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:48:14.361743 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981e4426-cb4c-4c8f-a729-df5aacbbcbea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:48:14.363359 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981e4426-cb4c-4c8f-a729-df5aacbbcbea admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:48:14.376399 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1118/2247] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:48:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:48:14.380802 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfafc8ca-8985-4ff2-91e6-9663da5744a6 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:48:14.391635 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfafc8ca-8985-4ff2-91e6-9663da5744a6 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:48:14.396848 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a08c02f-9942-4dfb-9604-0164382b4ef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:48:14.397756 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a08c02f-9942-4dfb-9604-0164382b4ef5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:48:14.398040 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a08c02f-9942-4dfb-9604-0164382b4ef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:48:14.398316 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a08c02f-9942-4dfb-9604-0164382b4ef5 admin admin] PATH_INFO: `/v3/projects/8dbfed972e0c4dcb88a9c4df5fcd6111/users/3b5bd48292d140eaaec23575e64d2542/roles/559942c4de1744169f8b5e0404ecde6b` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:48:14.413949 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a08c02f-9942-4dfb-9604-0164382b4ef5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8dbfed972e0c4dcb88a9c4df5fcd6111, user_id=3b5bd48292d140eaaec23575e64d2542, role_id=559942c4de1744169f8b5e0404ecde6b)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:48:14.416670 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a08c02f-9942-4dfb-9604-0164382b4ef5 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:48:14.418858 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:48:14.472073 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1130/2248] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:48:14 2022] PUT /identity/v3/projects/8dbfed972e0c4dcb88a9c4df5fcd6111/users/3b5bd48292d140eaaec23575e64d2542/roles/559942c4de1744169f8b5e0404ecde6b => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:48:14.482428 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8481d8-23dc-4ee4-9964-3b5e946c6e45 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:48:14.482721 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8481d8-23dc-4ee4-9964-3b5e946c6e45 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:48:14.483001 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8481d8-23dc-4ee4-9964-3b5e946c6e45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:48:14.512504 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-ac8481d8-23dc-4ee4-9964-3b5e946c6e45 None None] MFA Rules not processed for user `3b5bd48292d140eaaec23575e64d2542`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:48:14.512504 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-ac8481d8-23dc-4ee4-9964-3b5e946c6e45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:48:14.578118 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1119/2249] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:48:14 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:48:14.583797 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a08c02f-9942-4dfb-9604-0164382b4ef5 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:48:14.657054 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a08c02f-9942-4dfb-9604-0164382b4ef5 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:48:14.661892 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0098959-ca97-4a88-ac35-65a2794f79a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:48:14.662567 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0098959-ca97-4a88-ac35-65a2794f79a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:48:14.662855 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0098959-ca97-4a88-ac35-65a2794f79a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:48:14.663166 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0098959-ca97-4a88-ac35-65a2794f79a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:48:14.663875 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0098959-ca97-4a88-ac35-65a2794f79a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:48:14.665901 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0098959-ca97-4a88-ac35-65a2794f79a5 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:48:14.753036 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1131/2250] 10.222.0.81 () {60 vars in 1152 bytes} [Wed Jul 20 22:48:14 2022] POST /identity/v3/projects => generated 365 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:48:14.757081 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8481d8-23dc-4ee4-9964-3b5e946c6e45 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:48:14.767840 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8481d8-23dc-4ee4-9964-3b5e946c6e45 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:48:14.839004 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24e64af-2640-47c2-aca7-6473a379374c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:48:14.839696 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24e64af-2640-47c2-aca7-6473a379374c admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:48:14.839896 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24e64af-2640-47c2-aca7-6473a379374c admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:48:14.840179 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24e64af-2640-47c2-aca7-6473a379374c admin admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:48:14.840868 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24e64af-2640-47c2-aca7-6473a379374c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:48:14.842986 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24e64af-2640-47c2-aca7-6473a379374c admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:48:14.917344 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1120/2251] 10.222.0.81 () {60 vars in 1143 bytes} [Wed Jul 20 22:48:14 2022] POST /identity/v3/users => generated 289 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:48:14.921234 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0098959-ca97-4a88-ac35-65a2794f79a5 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:48:14.932348 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0098959-ca97-4a88-ac35-65a2794f79a5 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:48:14.937034 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e79f0cd-1c25-4f3f-9b22-4f5ad3ae198e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:48:14.937693 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e79f0cd-1c25-4f3f-9b22-4f5ad3ae198e admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:48:14.938163 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e79f0cd-1c25-4f3f-9b22-4f5ad3ae198e admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:48:14.938470 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e79f0cd-1c25-4f3f-9b22-4f5ad3ae198e admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:48:14.939230 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e79f0cd-1c25-4f3f-9b22-4f5ad3ae198e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:48:14.941401 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e79f0cd-1c25-4f3f-9b22-4f5ad3ae198e admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:48:14.955707 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1132/2252] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:48:14 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:48:14.960489 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24e64af-2640-47c2-aca7-6473a379374c admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:48:14.971237 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24e64af-2640-47c2-aca7-6473a379374c admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:48:14.975975 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa80163-e4a1-4b73-a534-977de651c84e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:48:14.976874 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa80163-e4a1-4b73-a534-977de651c84e admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:48:14.977161 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa80163-e4a1-4b73-a534-977de651c84e admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:48:14.977445 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa80163-e4a1-4b73-a534-977de651c84e admin admin] PATH_INFO: `/v3/projects/5b22fa68caed4746b98d5fd093c9b888/users/34011947896e45819b3f674fb8fc182d/roles/d89b6c92d35a401a8517f1d47f6be843` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:48:14.992056 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:48:14.995146 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa80163-e4a1-4b73-a534-977de651c84e admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b22fa68caed4746b98d5fd093c9b888, user_id=34011947896e45819b3f674fb8fc182d, role_id=d89b6c92d35a401a8517f1d47f6be843)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:48:14.997811 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa80163-e4a1-4b73-a534-977de651c84e admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:48:15.112948 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1121/2253] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:48:14 2022] PUT /identity/v3/projects/5b22fa68caed4746b98d5fd093c9b888/users/34011947896e45819b3f674fb8fc182d/roles/d89b6c92d35a401a8517f1d47f6be843 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:48:15.117041 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e79f0cd-1c25-4f3f-9b22-4f5ad3ae198e admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:48:15.134528 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e79f0cd-1c25-4f3f-9b22-4f5ad3ae198e admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:48:15.196227 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd946cc0-9788-4614-b759-83091510d366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:48:15.197335 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd946cc0-9788-4614-b759-83091510d366 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:48:15.197335 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd946cc0-9788-4614-b759-83091510d366 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:48:15.197515 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd946cc0-9788-4614-b759-83091510d366 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:48:15.198159 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd946cc0-9788-4614-b759-83091510d366 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:48:15.199885 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd946cc0-9788-4614-b759-83091510d366 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:48:15.209827 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1133/2254] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:48:15 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:48:15.214107 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa80163-e4a1-4b73-a534-977de651c84e admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:48:15.224475 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa80163-e4a1-4b73-a534-977de651c84e admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:48:15.229367 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d50cd7f-4c9e-4496-ac02-98cf9024b86a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:48:15.230411 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d50cd7f-4c9e-4496-ac02-98cf9024b86a admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:48:15.230785 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d50cd7f-4c9e-4496-ac02-98cf9024b86a admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:48:15.231090 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d50cd7f-4c9e-4496-ac02-98cf9024b86a admin admin] PATH_INFO: `/v3/projects/5b22fa68caed4746b98d5fd093c9b888/users/34011947896e45819b3f674fb8fc182d/roles/d89b6c92d35a401a8517f1d47f6be843` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:48:15.233237 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d50cd7f-4c9e-4496-ac02-98cf9024b86a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b22fa68caed4746b98d5fd093c9b888, user_id=34011947896e45819b3f674fb8fc182d, role_id=d89b6c92d35a401a8517f1d47f6be843)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:48:15.235986 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d50cd7f-4c9e-4496-ac02-98cf9024b86a admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:48:15.248246 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1122/2255] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:48:15 2022] PUT /identity/v3/projects/5b22fa68caed4746b98d5fd093c9b888/users/34011947896e45819b3f674fb8fc182d/roles/d89b6c92d35a401a8517f1d47f6be843 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:48:15.260220 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a5ca77-49bb-41ab-b951-e4198ef49787 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:48:15.268672 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a5ca77-49bb-41ab-b951-e4198ef49787 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:48:15.268672 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a5ca77-49bb-41ab-b951-e4198ef49787 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:48:15.300358 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-64a5ca77-49bb-41ab-b951-e4198ef49787 None None] MFA Rules not processed for user `34011947896e45819b3f674fb8fc182d`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:48:15.302124 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-64a5ca77-49bb-41ab-b951-e4198ef49787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:48:15.356589 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:48:15.376377 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1134/2256] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:48:15 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:48:15.387784 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d50cd7f-4c9e-4496-ac02-98cf9024b86a admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:48:15.389737 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-0d50cd7f-4c9e-4496-ac02-98cf9024b86a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:48:15.458896 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d50cd7f-4c9e-4496-ac02-98cf9024b86a admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:48:15.462970 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9317b05-4a47-47af-a416-8d3b8f516832 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecf4db289fba4e4a806198c62d31f7c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:48:15.463678 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9317b05-4a47-47af-a416-8d3b8f516832 service cinder] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:48:15.463960 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9317b05-4a47-47af-a416-8d3b8f516832 service cinder] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:48:15.464264 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9317b05-4a47-47af-a416-8d3b8f516832 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:48:15.474617 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9317b05-4a47-47af-a416-8d3b8f516832 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:48:15.476119 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9317b05-4a47-47af-a416-8d3b8f516832 service cinder] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:48:15.485750 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1123/2257] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:48:15 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:48:15.730622 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a5ca77-49bb-41ab-b951-e4198ef49787 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:48:15.740975 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a5ca77-49bb-41ab-b951-e4198ef49787 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:48:15.747594 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca14b1e-7635-4093-8ac0-d0b5885b24fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecf4db289fba4e4a806198c62d31f7c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:48:15.748291 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca14b1e-7635-4093-8ac0-d0b5885b24fd service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:48:15.748614 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca14b1e-7635-4093-8ac0-d0b5885b24fd service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:48:15.748892 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca14b1e-7635-4093-8ac0-d0b5885b24fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:48:15.807438 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca14b1e-7635-4093-8ac0-d0b5885b24fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:48:15.808905 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca14b1e-7635-4093-8ac0-d0b5885b24fd service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:48:15.817986 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1135/2258] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:48:15 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:48:25.481489 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=34,keystone:INSERT=2 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:48:25.813851 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=33 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:48:32.393961 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9317b05-4a47-47af-a416-8d3b8f516832 service cinder] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:48:32.395970 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-e9317b05-4a47-47af-a416-8d3b8f516832 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:48:32.469755 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9317b05-4a47-47af-a416-8d3b8f516832 service cinder] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:48:32.475694 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee896d2-302e-4cf2-82ca-acf1f74126af service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5500cebdd9d4f8e9aecb45d6c955878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:48:32.476365 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee896d2-302e-4cf2-82ca-acf1f74126af service placement] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:48:32.476662 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee896d2-302e-4cf2-82ca-acf1f74126af service placement] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:48:32.476945 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee896d2-302e-4cf2-82ca-acf1f74126af service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:48:32.478750 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-fee896d2-302e-4cf2-82ca-acf1f74126af service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:48:32.490665 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee896d2-302e-4cf2-82ca-acf1f74126af service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:48:32.492339 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee896d2-302e-4cf2-82ca-acf1f74126af service placement] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:48:32.502143 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1124/2259] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:48:32 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:48:42.498049 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=32 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:48:59.066165 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca14b1e-7635-4093-8ac0-d0b5885b24fd service cinder] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:48:59.128648 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca14b1e-7635-4093-8ac0-d0b5885b24fd service cinder] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:48:59.133207 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bdf811-5971-4c4e-af9b-d63cdbb6d8fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:48:59.134089 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bdf811-5971-4c4e-af9b-d63cdbb6d8fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:48:59.134369 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bdf811-5971-4c4e-af9b-d63cdbb6d8fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:48:59.134642 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bdf811-5971-4c4e-af9b-d63cdbb6d8fa admin admin] PATH_INFO: `/v3/users/3b5bd48292d140eaaec23575e64d2542` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:48:59.135851 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bdf811-5971-4c4e-af9b-d63cdbb6d8fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b5bd48292d140eaaec23575e64d2542)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:48:59.137771 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bdf811-5971-4c4e-af9b-d63cdbb6d8fa admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:48:59.316944 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-d7bdf811-5971-4c4e-af9b-d63cdbb6d8fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b5bd48292d140eaaec23575e64d2542'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:48:59.317860 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-d7bdf811-5971-4c4e-af9b-d63cdbb6d8fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b5bd48292d140eaaec23575e64d2542'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:48:59.318476 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-d7bdf811-5971-4c4e-af9b-d63cdbb6d8fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b5bd48292d140eaaec23575e64d2542'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:48:59.325644 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-d7bdf811-5971-4c4e-af9b-d63cdbb6d8fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b5bd48292d140eaaec23575e64d2542'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:48:59.400038 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-d7bdf811-5971-4c4e-af9b-d63cdbb6d8fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b5bd48292d140eaaec23575e64d2542'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:48:59.417731 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1136/2260] 10.222.0.81 () {58 vars in 1223 bytes} [Wed Jul 20 22:48:59 2022] DELETE /identity/v3/users/3b5bd48292d140eaaec23575e64d2542 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:48:59.422225 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7ecaa1-4b39-41ca-9599-d22517e9878f None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:48:59.422693 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7ecaa1-4b39-41ca-9599-d22517e9878f None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:48:59.422795 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7ecaa1-4b39-41ca-9599-d22517e9878f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:48:59.441271 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-4b7ecaa1-4b39-41ca-9599-d22517e9878f None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:48:59.443953 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-4b7ecaa1-4b39-41ca-9599-d22517e9878f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:48:59.500615 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1125/2261] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:48:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:48:59.510778 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bdf811-5971-4c4e-af9b-d63cdbb6d8fa admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:48:59.512191 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-d7bdf811-5971-4c4e-af9b-d63cdbb6d8fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:48:59.568909 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bdf811-5971-4c4e-af9b-d63cdbb6d8fa admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:48:59.572553 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea131ef5-882b-4e4b-8dbc-aa6de9a84cff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:48:59.573206 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea131ef5-882b-4e4b-8dbc-aa6de9a84cff service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:48:59.573495 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea131ef5-882b-4e4b-8dbc-aa6de9a84cff service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:48:59.573779 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea131ef5-882b-4e4b-8dbc-aa6de9a84cff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:48:59.582702 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea131ef5-882b-4e4b-8dbc-aa6de9a84cff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:48:59.584161 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea131ef5-882b-4e4b-8dbc-aa6de9a84cff service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:48:59.593496 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1137/2262] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:48:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:49:00.070841 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7ecaa1-4b39-41ca-9599-d22517e9878f None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:49:00.072328 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-4b7ecaa1-4b39-41ca-9599-d22517e9878f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:49:00.084856 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7ecaa1-4b39-41ca-9599-d22517e9878f None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:49:00.089338 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287c6992-1d03-4e5a-bbe8-b936bbed77fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:49:00.090294 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287c6992-1d03-4e5a-bbe8-b936bbed77fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:00.090586 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287c6992-1d03-4e5a-bbe8-b936bbed77fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:00.090871 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287c6992-1d03-4e5a-bbe8-b936bbed77fb admin admin] PATH_INFO: `/v3/projects/8dbfed972e0c4dcb88a9c4df5fcd6111` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:00.092625 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287c6992-1d03-4e5a-bbe8-b936bbed77fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=8dbfed972e0c4dcb88a9c4df5fcd6111)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:49:00.094579 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287c6992-1d03-4e5a-bbe8-b936bbed77fb admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:49:00.215168 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-287c6992-1d03-4e5a-bbe8-b936bbed77fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8dbfed972e0c4dcb88a9c4df5fcd6111', 'request_id': 'req-287c6992-1d03-4e5a-bbe8-b936bbed77fb'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:00.262540 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-287c6992-1d03-4e5a-bbe8-b936bbed77fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8dbfed972e0c4dcb88a9c4df5fcd6111', 'request_id': 'req-287c6992-1d03-4e5a-bbe8-b936bbed77fb'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:00.268456 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-287c6992-1d03-4e5a-bbe8-b936bbed77fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8dbfed972e0c4dcb88a9c4df5fcd6111', 'request_id': 'req-287c6992-1d03-4e5a-bbe8-b936bbed77fb'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:00.279809 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-287c6992-1d03-4e5a-bbe8-b936bbed77fb admin admin] The token cache is being invalidate because project 8dbfed972e0c4dcb88a9c4df5fcd6111 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:49:00.280101 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-287c6992-1d03-4e5a-bbe8-b936bbed77fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:00.280739 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-287c6992-1d03-4e5a-bbe8-b936bbed77fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:00.282274 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1126/2263] 10.222.0.81 () {58 vars in 1232 bytes} [Wed Jul 20 22:49:00 2022] DELETE /identity/v3/projects/8dbfed972e0c4dcb88a9c4df5fcd6111 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:49:00.286783 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea131ef5-882b-4e4b-8dbc-aa6de9a84cff service neutron] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:49:00.296414 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea131ef5-882b-4e4b-8dbc-aa6de9a84cff service neutron] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:49:00.314863 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:49:00.353142 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5111b422-d433-4861-9d95-e02938261356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:49:00.354024 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5111b422-d433-4861-9d95-e02938261356 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:00.354311 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5111b422-d433-4861-9d95-e02938261356 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:00.354586 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5111b422-d433-4861-9d95-e02938261356 admin admin] PATH_INFO: `/v3/users/34011947896e45819b3f674fb8fc182d` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:00.355807 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5111b422-d433-4861-9d95-e02938261356 admin admin] RBAC: Authorizing `identity:delete_user(user_id=34011947896e45819b3f674fb8fc182d)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:49:00.357805 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5111b422-d433-4861-9d95-e02938261356 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:49:00.572718 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-5111b422-d433-4861-9d95-e02938261356 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34011947896e45819b3f674fb8fc182d'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:00.573414 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-5111b422-d433-4861-9d95-e02938261356 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34011947896e45819b3f674fb8fc182d'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:00.574028 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-5111b422-d433-4861-9d95-e02938261356 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34011947896e45819b3f674fb8fc182d'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:00.580918 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-5111b422-d433-4861-9d95-e02938261356 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34011947896e45819b3f674fb8fc182d'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:00.679758 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-5111b422-d433-4861-9d95-e02938261356 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34011947896e45819b3f674fb8fc182d'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:00.699470 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1138/2264] 10.222.0.81 () {58 vars in 1223 bytes} [Wed Jul 20 22:49:00 2022] DELETE /identity/v3/users/34011947896e45819b3f674fb8fc182d => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:49:01.156010 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287c6992-1d03-4e5a-bbe8-b936bbed77fb admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:49:01.167204 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-287c6992-1d03-4e5a-bbe8-b936bbed77fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:49:01.170989 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287c6992-1d03-4e5a-bbe8-b936bbed77fb admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:49:01.231202 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7accd0-4e30-4d0e-a0e7-477113c678ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:49:01.232088 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7accd0-4e30-4d0e-a0e7-477113c678ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:01.232369 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7accd0-4e30-4d0e-a0e7-477113c678ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:01.232780 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7accd0-4e30-4d0e-a0e7-477113c678ec admin admin] PATH_INFO: `/v3/projects/5b22fa68caed4746b98d5fd093c9b888` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:01.233918 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7accd0-4e30-4d0e-a0e7-477113c678ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b22fa68caed4746b98d5fd093c9b888)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:49:01.235860 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7accd0-4e30-4d0e-a0e7-477113c678ec admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:49:01.384640 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:49:01.401172 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-0b7accd0-4e30-4d0e-a0e7-477113c678ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b22fa68caed4746b98d5fd093c9b888', 'request_id': 'req-0b7accd0-4e30-4d0e-a0e7-477113c678ec'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:01.497033 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-0b7accd0-4e30-4d0e-a0e7-477113c678ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b22fa68caed4746b98d5fd093c9b888', 'request_id': 'req-0b7accd0-4e30-4d0e-a0e7-477113c678ec'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:01.503058 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-0b7accd0-4e30-4d0e-a0e7-477113c678ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b22fa68caed4746b98d5fd093c9b888', 'request_id': 'req-0b7accd0-4e30-4d0e-a0e7-477113c678ec'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:01.514736 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-0b7accd0-4e30-4d0e-a0e7-477113c678ec admin admin] The token cache is being invalidate because project 5b22fa68caed4746b98d5fd093c9b888 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:49:01.515059 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-0b7accd0-4e30-4d0e-a0e7-477113c678ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:01.515702 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-0b7accd0-4e30-4d0e-a0e7-477113c678ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:01.517242 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1127/2265] 10.222.0.81 () {58 vars in 1232 bytes} [Wed Jul 20 22:49:01 2022] DELETE /identity/v3/projects/5b22fa68caed4746b98d5fd093c9b888 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:49:01.522208 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae411ec0-7e83-469d-a6c9-ef31a0c46b44 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:01.522668 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae411ec0-7e83-469d-a6c9-ef31a0c46b44 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:01.522796 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae411ec0-7e83-469d-a6c9-ef31a0c46b44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:01.540254 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-ae411ec0-7e83-469d-a6c9-ef31a0c46b44 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:49:01.541526 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-ae411ec0-7e83-469d-a6c9-ef31a0c46b44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:49:01.604552 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1139/2266] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:49:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:49:01.613102 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dafebd-e706-47c4-8e05-5bc8b1c56cc9 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:01.613387 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dafebd-e706-47c4-8e05-5bc8b1c56cc9 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:01.613664 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dafebd-e706-47c4-8e05-5bc8b1c56cc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:01.632799 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-f7dafebd-e706-47c4-8e05-5bc8b1c56cc9 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:49:01.634029 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-f7dafebd-e706-47c4-8e05-5bc8b1c56cc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:49:01.642329 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1128/2267] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:49:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:49:01.646660 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae411ec0-7e83-469d-a6c9-ef31a0c46b44 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:49:01.656727 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae411ec0-7e83-469d-a6c9-ef31a0c46b44 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:49:01.661593 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbdf1de-4a31-48a0-afc2-410ac1e88935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:49:01.662293 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbdf1de-4a31-48a0-afc2-410ac1e88935 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:01.662585 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbdf1de-4a31-48a0-afc2-410ac1e88935 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:01.665497 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbdf1de-4a31-48a0-afc2-410ac1e88935 admin admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:01.665497 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbdf1de-4a31-48a0-afc2-410ac1e88935 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:49:01.665497 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbdf1de-4a31-48a0-afc2-410ac1e88935 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:49:01.665763 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbdf1de-4a31-48a0-afc2-410ac1e88935 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:49:01.678030 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1140/2268] 10.222.0.81 () {58 vars in 1152 bytes} [Wed Jul 20 22:49:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:49:01.682469 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dafebd-e706-47c4-8e05-5bc8b1c56cc9 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:49:01.693774 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dafebd-e706-47c4-8e05-5bc8b1c56cc9 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:49:01.698700 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e6c9ec-1231-48a6-b34b-9416d9fae08c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:49:01.699408 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e6c9ec-1231-48a6-b34b-9416d9fae08c admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:01.699694 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e6c9ec-1231-48a6-b34b-9416d9fae08c admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:01.699974 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e6c9ec-1231-48a6-b34b-9416d9fae08c admin admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:01.700690 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e6c9ec-1231-48a6-b34b-9416d9fae08c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:49:01.702594 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e6c9ec-1231-48a6-b34b-9416d9fae08c admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:49:01.802569 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1129/2269] 10.222.0.81 () {60 vars in 1152 bytes} [Wed Jul 20 22:49:01 2022] POST /identity/v3/projects => generated 385 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:49:01.806891 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbdf1de-4a31-48a0-afc2-410ac1e88935 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:49:01.817445 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbdf1de-4a31-48a0-afc2-410ac1e88935 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:49:01.844029 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:49:01.872799 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124eb1ef-bde4-48da-83c9-c90fc0c0731a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:49:01.873473 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124eb1ef-bde4-48da-83c9-c90fc0c0731a admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:01.873764 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124eb1ef-bde4-48da-83c9-c90fc0c0731a admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:01.874046 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124eb1ef-bde4-48da-83c9-c90fc0c0731a admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:01.874730 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124eb1ef-bde4-48da-83c9-c90fc0c0731a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:49:01.876683 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124eb1ef-bde4-48da-83c9-c90fc0c0731a admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:49:01.949550 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1141/2270] 10.222.0.81 () {60 vars in 1143 bytes} [Wed Jul 20 22:49:01 2022] POST /identity/v3/users => generated 293 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:49:01.955755 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e6c9ec-1231-48a6-b34b-9416d9fae08c admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:49:01.962923 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e6c9ec-1231-48a6-b34b-9416d9fae08c admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:49:01.967748 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899e44a7-8957-44c0-b2b7-b51c3cbcfa05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:49:01.968442 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899e44a7-8957-44c0-b2b7-b51c3cbcfa05 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:01.968841 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899e44a7-8957-44c0-b2b7-b51c3cbcfa05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:01.969119 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899e44a7-8957-44c0-b2b7-b51c3cbcfa05 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:01.969841 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899e44a7-8957-44c0-b2b7-b51c3cbcfa05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:49:01.971486 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899e44a7-8957-44c0-b2b7-b51c3cbcfa05 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:49:01.982393 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1130/2271] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:49:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:49:01.986489 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124eb1ef-bde4-48da-83c9-c90fc0c0731a admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:49:01.996778 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124eb1ef-bde4-48da-83c9-c90fc0c0731a admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:49:02.002224 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dedcb8f-1c8f-430c-b990-48fe2f7c03c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:49:02.003507 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dedcb8f-1c8f-430c-b990-48fe2f7c03c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:02.007213 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dedcb8f-1c8f-430c-b990-48fe2f7c03c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:02.007213 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dedcb8f-1c8f-430c-b990-48fe2f7c03c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:02.007213 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dedcb8f-1c8f-430c-b990-48fe2f7c03c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:49:02.007213 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dedcb8f-1c8f-430c-b990-48fe2f7c03c0 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:49:02.017547 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1142/2272] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:49:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:49:02.022197 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899e44a7-8957-44c0-b2b7-b51c3cbcfa05 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:49:02.033129 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899e44a7-8957-44c0-b2b7-b51c3cbcfa05 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:49:02.038381 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d5deaf-3abe-44c1-9b11-56aa4d4d1f7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:49:02.039303 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d5deaf-3abe-44c1-9b11-56aa4d4d1f7b admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:02.039590 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d5deaf-3abe-44c1-9b11-56aa4d4d1f7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:02.039871 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d5deaf-3abe-44c1-9b11-56aa4d4d1f7b admin admin] PATH_INFO: `/v3/projects/df6ea6f7bee542f3b7edc57c00ec2e91/users/a118ee578d6448ab99a68a77f09a01dc/roles/559942c4de1744169f8b5e0404ecde6b` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:02.054950 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d5deaf-3abe-44c1-9b11-56aa4d4d1f7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=df6ea6f7bee542f3b7edc57c00ec2e91, user_id=a118ee578d6448ab99a68a77f09a01dc, role_id=559942c4de1744169f8b5e0404ecde6b)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:49:02.057614 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d5deaf-3abe-44c1-9b11-56aa4d4d1f7b admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:49:02.137464 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1131/2273] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:49:02 2022] PUT /identity/v3/projects/df6ea6f7bee542f3b7edc57c00ec2e91/users/a118ee578d6448ab99a68a77f09a01dc/roles/559942c4de1744169f8b5e0404ecde6b => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:49:02.148345 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e364ee6b-f69d-402c-8ce6-0fbdc603e33d None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:02.148787 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e364ee6b-f69d-402c-8ce6-0fbdc603e33d None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:02.149131 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e364ee6b-f69d-402c-8ce6-0fbdc603e33d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:02.176464 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-e364ee6b-f69d-402c-8ce6-0fbdc603e33d None None] MFA Rules not processed for user `a118ee578d6448ab99a68a77f09a01dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:49:02.177633 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-e364ee6b-f69d-402c-8ce6-0fbdc603e33d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:49:02.241038 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1143/2274] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:49:02 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:49:02.245510 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d5deaf-3abe-44c1-9b11-56aa4d4d1f7b admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:49:02.307059 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d5deaf-3abe-44c1-9b11-56aa4d4d1f7b admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:49:02.311833 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9089a6c-659f-40b8-a073-d4843249e66c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:49:02.312496 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9089a6c-659f-40b8-a073-d4843249e66c admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:02.312834 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9089a6c-659f-40b8-a073-d4843249e66c admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:02.313118 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9089a6c-659f-40b8-a073-d4843249e66c admin admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:02.313850 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9089a6c-659f-40b8-a073-d4843249e66c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:49:02.316348 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9089a6c-659f-40b8-a073-d4843249e66c admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:49:02.353439 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e364ee6b-f69d-402c-8ce6-0fbdc603e33d None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:49:02.354836 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-e364ee6b-f69d-402c-8ce6-0fbdc603e33d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:49:02.368900 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e364ee6b-f69d-402c-8ce6-0fbdc603e33d None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:49:02.377602 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee60c46c-17cf-4247-8ff9-b9a395b84594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:49:02.377602 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee60c46c-17cf-4247-8ff9-b9a395b84594 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:02.377602 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee60c46c-17cf-4247-8ff9-b9a395b84594 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:02.377602 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee60c46c-17cf-4247-8ff9-b9a395b84594 admin admin] PATH_INFO: `/v3/users/661e510ad57b48a3b462577e4275474c` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:02.378003 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee60c46c-17cf-4247-8ff9-b9a395b84594 admin admin] RBAC: Authorizing `identity:delete_user(user_id=661e510ad57b48a3b462577e4275474c)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:49:02.378816 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee60c46c-17cf-4247-8ff9-b9a395b84594 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:49:02.404788 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1132/2275] 10.222.0.81 () {60 vars in 1152 bytes} [Wed Jul 20 22:49:02 2022] POST /identity/v3/projects => generated 385 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:49:02.406357 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9089a6c-659f-40b8-a073-d4843249e66c admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:49:02.444816 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:49:02.466799 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9089a6c-659f-40b8-a073-d4843249e66c admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:49:02.472764 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170d03ea-d789-4eb5-be8d-6776485d9b33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:49:02.473529 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170d03ea-d789-4eb5-be8d-6776485d9b33 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:02.473811 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170d03ea-d789-4eb5-be8d-6776485d9b33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:02.474089 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170d03ea-d789-4eb5-be8d-6776485d9b33 admin admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:02.474773 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170d03ea-d789-4eb5-be8d-6776485d9b33 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:49:02.476809 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170d03ea-d789-4eb5-be8d-6776485d9b33 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:49:02.619545 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-ee60c46c-17cf-4247-8ff9-b9a395b84594 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '661e510ad57b48a3b462577e4275474c'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:02.620222 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-ee60c46c-17cf-4247-8ff9-b9a395b84594 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '661e510ad57b48a3b462577e4275474c'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:02.620848 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-ee60c46c-17cf-4247-8ff9-b9a395b84594 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '661e510ad57b48a3b462577e4275474c'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:02.623188 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1133/2276] 10.222.0.81 () {60 vars in 1143 bytes} [Wed Jul 20 22:49:02 2022] POST /identity/v3/users => generated 299 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 22:49:02.628457 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170d03ea-d789-4eb5-be8d-6776485d9b33 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:49:02.628614 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-ee60c46c-17cf-4247-8ff9-b9a395b84594 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '661e510ad57b48a3b462577e4275474c'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:02.629005 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-170d03ea-d789-4eb5-be8d-6776485d9b33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:49:02.641008 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170d03ea-d789-4eb5-be8d-6776485d9b33 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:49:02.645744 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f270527e-5c2c-4b36-a688-237c29a2baf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:49:02.646493 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f270527e-5c2c-4b36-a688-237c29a2baf6 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:02.646900 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f270527e-5c2c-4b36-a688-237c29a2baf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:02.647309 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f270527e-5c2c-4b36-a688-237c29a2baf6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:02.648040 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f270527e-5c2c-4b36-a688-237c29a2baf6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:49:02.649768 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f270527e-5c2c-4b36-a688-237c29a2baf6 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:49:02.660708 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1134/2277] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:49:02 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 22:49:02.664893 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f270527e-5c2c-4b36-a688-237c29a2baf6 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:49:02.674405 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f270527e-5c2c-4b36-a688-237c29a2baf6 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:49:02.680294 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ea092a-1cf5-4f17-9fde-e4cd6e342d13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:49:02.683039 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ea092a-1cf5-4f17-9fde-e4cd6e342d13 admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:02.683039 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ea092a-1cf5-4f17-9fde-e4cd6e342d13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:02.683039 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ea092a-1cf5-4f17-9fde-e4cd6e342d13 admin admin] PATH_INFO: `/v3/projects/bf0e589e4c3946ea8e02bf7f08a7433b/users/de86f19a0ee1427bbec6b186776cf963/roles/d89b6c92d35a401a8517f1d47f6be843` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:02.698193 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ea092a-1cf5-4f17-9fde-e4cd6e342d13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf0e589e4c3946ea8e02bf7f08a7433b, user_id=de86f19a0ee1427bbec6b186776cf963, role_id=d89b6c92d35a401a8517f1d47f6be843)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:49:02.700830 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ea092a-1cf5-4f17-9fde-e4cd6e342d13 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:49:02.702892 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-ee60c46c-17cf-4247-8ff9-b9a395b84594 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '661e510ad57b48a3b462577e4275474c'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:02.721757 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1144/2278] 10.222.0.81 () {58 vars in 1223 bytes} [Wed Jul 20 22:49:02 2022] DELETE /identity/v3/users/661e510ad57b48a3b462577e4275474c => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 22:49:02.726110 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e376cc63-c331-4ff6-997f-6a8fea0b9e33 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:02.726409 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e376cc63-c331-4ff6-997f-6a8fea0b9e33 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:02.726697 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e376cc63-c331-4ff6-997f-6a8fea0b9e33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:02.742885 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-e376cc63-c331-4ff6-997f-6a8fea0b9e33 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:49:02.744131 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-e376cc63-c331-4ff6-997f-6a8fea0b9e33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:49:02.748056 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:49:02.788898 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1135/2279] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:49:02 2022] PUT /identity/v3/projects/bf0e589e4c3946ea8e02bf7f08a7433b/users/de86f19a0ee1427bbec6b186776cf963/roles/d89b6c92d35a401a8517f1d47f6be843 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 22:49:02.792833 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ea092a-1cf5-4f17-9fde-e4cd6e342d13 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:49:02.806813 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1145/2280] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:49:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 22:49:02.817712 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e376cc63-c331-4ff6-997f-6a8fea0b9e33 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:49:02.819128 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-e376cc63-c331-4ff6-997f-6a8fea0b9e33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:49:02.855278 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ea092a-1cf5-4f17-9fde-e4cd6e342d13 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:49:02.860172 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300eb90d-9026-49ed-aef6-e6d41ae60036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:49:02.860844 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300eb90d-9026-49ed-aef6-e6d41ae60036 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:02.861158 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300eb90d-9026-49ed-aef6-e6d41ae60036 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:02.861442 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300eb90d-9026-49ed-aef6-e6d41ae60036 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:02.862168 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300eb90d-9026-49ed-aef6-e6d41ae60036 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:49:02.864279 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300eb90d-9026-49ed-aef6-e6d41ae60036 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:49:02.874551 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1136/2281] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:49:02 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 22:49:02.877517 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e376cc63-c331-4ff6-997f-6a8fea0b9e33 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:49:02.879451 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300eb90d-9026-49ed-aef6-e6d41ae60036 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:49:02.889342 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300eb90d-9026-49ed-aef6-e6d41ae60036 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:49:02.894024 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ed021c-bf96-4791-b373-30eac08aa630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:49:02.894913 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ed021c-bf96-4791-b373-30eac08aa630 admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:02.895268 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ed021c-bf96-4791-b373-30eac08aa630 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:02.895521 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ed021c-bf96-4791-b373-30eac08aa630 admin admin] PATH_INFO: `/v3/projects/bf0e589e4c3946ea8e02bf7f08a7433b/users/de86f19a0ee1427bbec6b186776cf963/roles/d89b6c92d35a401a8517f1d47f6be843` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:02.897518 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ed021c-bf96-4791-b373-30eac08aa630 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf0e589e4c3946ea8e02bf7f08a7433b, user_id=de86f19a0ee1427bbec6b186776cf963, role_id=d89b6c92d35a401a8517f1d47f6be843)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:49:02.900199 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ed021c-bf96-4791-b373-30eac08aa630 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:49:02.910898 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1137/2282] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:49:02 2022] PUT /identity/v3/projects/bf0e589e4c3946ea8e02bf7f08a7433b/users/de86f19a0ee1427bbec6b186776cf963/roles/d89b6c92d35a401a8517f1d47f6be843 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:49:02.922149 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13be7c29-ab74-4792-8d00-3cdefcbba188 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:02.922492 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13be7c29-ab74-4792-8d00-3cdefcbba188 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:02.922775 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13be7c29-ab74-4792-8d00-3cdefcbba188 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:02.926812 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42adb770-d07e-476f-b9db-35b7f68460f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:49:02.927544 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42adb770-d07e-476f-b9db-35b7f68460f8 service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:02.927847 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42adb770-d07e-476f-b9db-35b7f68460f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:02.928123 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42adb770-d07e-476f-b9db-35b7f68460f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:02.937695 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42adb770-d07e-476f-b9db-35b7f68460f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:49:02.939175 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42adb770-d07e-476f-b9db-35b7f68460f8 service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:49:02.949689 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-13be7c29-ab74-4792-8d00-3cdefcbba188 None None] MFA Rules not processed for user `de86f19a0ee1427bbec6b186776cf963`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:49:02.949883 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1146/2283] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:49:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 22:49:02.952323 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-13be7c29-ab74-4792-8d00-3cdefcbba188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:49:03.027700 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:49:03.032863 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1138/2284] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:49:02 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 22:49:03.043910 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42adb770-d07e-476f-b9db-35b7f68460f8 service neutron] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:49:03.045454 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-42adb770-d07e-476f-b9db-35b7f68460f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:49:03.101426 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42adb770-d07e-476f-b9db-35b7f68460f8 service neutron] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:49:03.120813 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:49:03.148826 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea00ef1-0a61-45de-a300-fd47f3db3c78 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecf4db289fba4e4a806198c62d31f7c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:49:03.149480 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea00ef1-0a61-45de-a300-fd47f3db3c78 service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:03.149771 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea00ef1-0a61-45de-a300-fd47f3db3c78 service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:03.150053 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea00ef1-0a61-45de-a300-fd47f3db3c78 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:03.159275 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea00ef1-0a61-45de-a300-fd47f3db3c78 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:49:03.160699 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea00ef1-0a61-45de-a300-fd47f3db3c78 service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:49:03.170314 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1147/2285] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:49:03 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:49:03.418689 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13be7c29-ab74-4792-8d00-3cdefcbba188 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:49:03.420195 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-13be7c29-ab74-4792-8d00-3cdefcbba188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:49:03.496613 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13be7c29-ab74-4792-8d00-3cdefcbba188 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:49:03.500917 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9217a3-0675-42ac-a205-8ca84f2f354a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:49:03.501969 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9217a3-0675-42ac-a205-8ca84f2f354a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:03.502262 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9217a3-0675-42ac-a205-8ca84f2f354a admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:03.502547 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9217a3-0675-42ac-a205-8ca84f2f354a admin admin] PATH_INFO: `/v3/projects/6d6195ade06843318b16e5f12be23c38` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:03.504350 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9217a3-0675-42ac-a205-8ca84f2f354a admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d6195ade06843318b16e5f12be23c38)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:49:03.506269 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9217a3-0675-42ac-a205-8ca84f2f354a admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:49:03.663124 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-9b9217a3-0675-42ac-a205-8ca84f2f354a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d6195ade06843318b16e5f12be23c38', 'request_id': 'req-9b9217a3-0675-42ac-a205-8ca84f2f354a'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:03.793990 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-9b9217a3-0675-42ac-a205-8ca84f2f354a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d6195ade06843318b16e5f12be23c38', 'request_id': 'req-9b9217a3-0675-42ac-a205-8ca84f2f354a'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:03.800654 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-9b9217a3-0675-42ac-a205-8ca84f2f354a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d6195ade06843318b16e5f12be23c38', 'request_id': 'req-9b9217a3-0675-42ac-a205-8ca84f2f354a'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:03.812071 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-9b9217a3-0675-42ac-a205-8ca84f2f354a admin admin] The token cache is being invalidate because project 6d6195ade06843318b16e5f12be23c38 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:49:03.824704 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-9b9217a3-0675-42ac-a205-8ca84f2f354a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:03.824704 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-9b9217a3-0675-42ac-a205-8ca84f2f354a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:03.824704 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1139/2286] 10.222.0.81 () {58 vars in 1232 bytes} [Wed Jul 20 22:49:03 2022] DELETE /identity/v3/projects/6d6195ade06843318b16e5f12be23c38 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:49:03.825434 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e26fa7a-7fc1-4f63-b960-d3682c8e8309 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:03.825434 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e26fa7a-7fc1-4f63-b960-d3682c8e8309 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:03.825434 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e26fa7a-7fc1-4f63-b960-d3682c8e8309 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:03.840393 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-7e26fa7a-7fc1-4f63-b960-d3682c8e8309 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:49:03.841657 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-7e26fa7a-7fc1-4f63-b960-d3682c8e8309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:49:03.903953 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1148/2287] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:49:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:49:03.912089 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c719aa-bb72-4137-9c55-1cf984c37ac7 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:03.912386 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c719aa-bb72-4137-9c55-1cf984c37ac7 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:03.912781 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c719aa-bb72-4137-9c55-1cf984c37ac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:03.932644 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-e5c719aa-bb72-4137-9c55-1cf984c37ac7 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:49:03.933876 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-e5c719aa-bb72-4137-9c55-1cf984c37ac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:49:03.942922 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1140/2288] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:49:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:49:03.946852 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e26fa7a-7fc1-4f63-b960-d3682c8e8309 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:49:03.957312 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e26fa7a-7fc1-4f63-b960-d3682c8e8309 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:49:03.962214 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff786ba-e04b-475e-a476-e38479107cf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:49:03.962911 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff786ba-e04b-475e-a476-e38479107cf4 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:03.963268 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff786ba-e04b-475e-a476-e38479107cf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:03.963506 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff786ba-e04b-475e-a476-e38479107cf4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:03.964253 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff786ba-e04b-475e-a476-e38479107cf4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:49:03.964563 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff786ba-e04b-475e-a476-e38479107cf4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:49:03.966188 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff786ba-e04b-475e-a476-e38479107cf4 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:49:03.980298 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1149/2289] 10.222.0.81 () {58 vars in 1152 bytes} [Wed Jul 20 22:49:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:49:03.985191 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c719aa-bb72-4137-9c55-1cf984c37ac7 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:49:03.995437 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c719aa-bb72-4137-9c55-1cf984c37ac7 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:49:04.004550 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce6670c-b367-424a-9a43-3622cfd4cae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:49:04.005253 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce6670c-b367-424a-9a43-3622cfd4cae0 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:04.005541 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce6670c-b367-424a-9a43-3622cfd4cae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:04.005823 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce6670c-b367-424a-9a43-3622cfd4cae0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:04.019483 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce6670c-b367-424a-9a43-3622cfd4cae0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:49:04.019483 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce6670c-b367-424a-9a43-3622cfd4cae0 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:49:04.182291 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1141/2290] 10.222.0.81 () {60 vars in 1152 bytes} [Wed Jul 20 22:49:03 2022] POST /identity/v3/projects => generated 383 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:49:04.183759 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff786ba-e04b-475e-a476-e38479107cf4 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:49:04.194477 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff786ba-e04b-475e-a476-e38479107cf4 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:49:04.249960 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826b1150-a4eb-4c98-a0f5-a019ac8b556e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:49:04.250712 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826b1150-a4eb-4c98-a0f5-a019ac8b556e admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:04.251035 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826b1150-a4eb-4c98-a0f5-a019ac8b556e admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:04.251316 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826b1150-a4eb-4c98-a0f5-a019ac8b556e admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:04.252023 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826b1150-a4eb-4c98-a0f5-a019ac8b556e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:49:04.254802 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826b1150-a4eb-4c98-a0f5-a019ac8b556e admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:49:04.366303 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1150/2291] 10.222.0.81 () {60 vars in 1143 bytes} [Wed Jul 20 22:49:04 2022] POST /identity/v3/users => generated 292 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:49:04.370661 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce6670c-b367-424a-9a43-3622cfd4cae0 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:49:04.381508 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce6670c-b367-424a-9a43-3622cfd4cae0 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:49:04.386113 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22749a77-4f5f-40b7-a34d-64827d53931e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:49:04.386769 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22749a77-4f5f-40b7-a34d-64827d53931e admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:04.387091 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22749a77-4f5f-40b7-a34d-64827d53931e admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:04.390820 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22749a77-4f5f-40b7-a34d-64827d53931e admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:04.390820 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22749a77-4f5f-40b7-a34d-64827d53931e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:49:04.390820 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22749a77-4f5f-40b7-a34d-64827d53931e admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:49:04.403619 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1142/2292] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:49:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:49:04.404771 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826b1150-a4eb-4c98-a0f5-a019ac8b556e admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:49:04.415277 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826b1150-a4eb-4c98-a0f5-a019ac8b556e admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:49:04.425519 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce904092-63e3-42d5-b790-37b2831b238e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:49:04.425519 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce904092-63e3-42d5-b790-37b2831b238e admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:04.425519 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce904092-63e3-42d5-b790-37b2831b238e admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:04.425519 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce904092-63e3-42d5-b790-37b2831b238e admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:04.426037 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce904092-63e3-42d5-b790-37b2831b238e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:49:04.426037 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce904092-63e3-42d5-b790-37b2831b238e admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:49:04.441778 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1151/2293] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:49:04 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:49:04.443082 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22749a77-4f5f-40b7-a34d-64827d53931e admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:49:04.465997 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22749a77-4f5f-40b7-a34d-64827d53931e admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:49:04.465997 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5b21ff-cf10-40b5-b6fc-52a5f72cef58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:49:04.465997 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5b21ff-cf10-40b5-b6fc-52a5f72cef58 admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:04.465997 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5b21ff-cf10-40b5-b6fc-52a5f72cef58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:04.466644 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5b21ff-cf10-40b5-b6fc-52a5f72cef58 admin admin] PATH_INFO: `/v3/projects/650d54a3145d42748ed81457cdd1eb34/users/1f237b9adae24d1ca2262d89eb000de4/roles/559942c4de1744169f8b5e0404ecde6b` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:04.479032 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5b21ff-cf10-40b5-b6fc-52a5f72cef58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=650d54a3145d42748ed81457cdd1eb34, user_id=1f237b9adae24d1ca2262d89eb000de4, role_id=559942c4de1744169f8b5e0404ecde6b)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:49:04.481698 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5b21ff-cf10-40b5-b6fc-52a5f72cef58 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:49:04.572406 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1143/2294] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:49:04 2022] PUT /identity/v3/projects/650d54a3145d42748ed81457cdd1eb34/users/1f237b9adae24d1ca2262d89eb000de4/roles/559942c4de1744169f8b5e0404ecde6b => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 22:49:04.577773 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711adc87-a7aa-4f8e-9deb-60d6cff5d68e None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:04.578071 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711adc87-a7aa-4f8e-9deb-60d6cff5d68e None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:04.578357 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711adc87-a7aa-4f8e-9deb-60d6cff5d68e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:04.596804 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-711adc87-a7aa-4f8e-9deb-60d6cff5d68e None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:49:04.598160 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-711adc87-a7aa-4f8e-9deb-60d6cff5d68e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:49:04.621896 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5b21ff-cf10-40b5-b6fc-52a5f72cef58 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:49:04.623771 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-ab5b21ff-cf10-40b5-b6fc-52a5f72cef58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:49:04.633622 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:49:04.652007 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:49:04.663368 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1152/2295] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:49:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:49:04.673247 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711adc87-a7aa-4f8e-9deb-60d6cff5d68e None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:49:04.674588 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-711adc87-a7aa-4f8e-9deb-60d6cff5d68e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:49:04.689690 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5b21ff-cf10-40b5-b6fc-52a5f72cef58 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:49:04.695704 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c8ec9c-3e0e-40b4-ba56-d1d857f07e4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:49:04.696608 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c8ec9c-3e0e-40b4-ba56-d1d857f07e4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:04.696909 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c8ec9c-3e0e-40b4-ba56-d1d857f07e4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:04.697226 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c8ec9c-3e0e-40b4-ba56-d1d857f07e4b admin admin] PATH_INFO: `/v3/users/a118ee578d6448ab99a68a77f09a01dc` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:04.698579 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c8ec9c-3e0e-40b4-ba56-d1d857f07e4b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a118ee578d6448ab99a68a77f09a01dc)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:49:04.701506 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c8ec9c-3e0e-40b4-ba56-d1d857f07e4b admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:49:04.752158 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711adc87-a7aa-4f8e-9deb-60d6cff5d68e None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:49:04.756558 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4526e1-813a-4e5a-aae2-57ca292cd1b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:49:04.757210 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4526e1-813a-4e5a-aae2-57ca292cd1b4 service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:04.757493 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4526e1-813a-4e5a-aae2-57ca292cd1b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:04.757770 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4526e1-813a-4e5a-aae2-57ca292cd1b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:04.767429 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4526e1-813a-4e5a-aae2-57ca292cd1b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:49:04.769102 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4526e1-813a-4e5a-aae2-57ca292cd1b4 service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:49:04.779127 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1153/2296] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:49:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 22:49:04.910689 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-14c8ec9c-3e0e-40b4-ba56-d1d857f07e4b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a118ee578d6448ab99a68a77f09a01dc'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:04.911387 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-14c8ec9c-3e0e-40b4-ba56-d1d857f07e4b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a118ee578d6448ab99a68a77f09a01dc'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:04.912075 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-14c8ec9c-3e0e-40b4-ba56-d1d857f07e4b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a118ee578d6448ab99a68a77f09a01dc'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:04.918915 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-14c8ec9c-3e0e-40b4-ba56-d1d857f07e4b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a118ee578d6448ab99a68a77f09a01dc'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:04.982206 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-14c8ec9c-3e0e-40b4-ba56-d1d857f07e4b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a118ee578d6448ab99a68a77f09a01dc'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:04.999993 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1144/2297] 10.222.0.81 () {58 vars in 1223 bytes} [Wed Jul 20 22:49:04 2022] DELETE /identity/v3/users/a118ee578d6448ab99a68a77f09a01dc => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 22:49:05.004990 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa25453b-f4bb-4ca0-a558-7aa27c2f53aa None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:05.005475 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa25453b-f4bb-4ca0-a558-7aa27c2f53aa None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:05.005598 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa25453b-f4bb-4ca0-a558-7aa27c2f53aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:05.023148 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-fa25453b-f4bb-4ca0-a558-7aa27c2f53aa None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:49:05.024381 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-fa25453b-f4bb-4ca0-a558-7aa27c2f53aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:49:05.089212 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1154/2298] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:49:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:49:05.098817 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c8ec9c-3e0e-40b4-ba56-d1d857f07e4b admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:49:05.102143 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-14c8ec9c-3e0e-40b4-ba56-d1d857f07e4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:49:05.174036 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c8ec9c-3e0e-40b4-ba56-d1d857f07e4b admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:49:05.178622 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fe6156-cabd-430b-a2db-1e7502a4e448 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:49:05.179382 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fe6156-cabd-430b-a2db-1e7502a4e448 service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:05.179736 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fe6156-cabd-430b-a2db-1e7502a4e448 service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:05.183156 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fe6156-cabd-430b-a2db-1e7502a4e448 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:05.198098 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fe6156-cabd-430b-a2db-1e7502a4e448 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:49:05.199657 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fe6156-cabd-430b-a2db-1e7502a4e448 service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:49:05.210612 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1145/2299] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:49:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:49:05.847337 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa25453b-f4bb-4ca0-a558-7aa27c2f53aa None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:49:05.849256 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-fa25453b-f4bb-4ca0-a558-7aa27c2f53aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:49:05.880356 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa25453b-f4bb-4ca0-a558-7aa27c2f53aa None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:49:05.901208 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5908d52-4710-42fc-8ad4-32ee9137e4ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:49:05.902824 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5908d52-4710-42fc-8ad4-32ee9137e4ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:05.903306 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5908d52-4710-42fc-8ad4-32ee9137e4ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:05.903749 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5908d52-4710-42fc-8ad4-32ee9137e4ac admin admin] PATH_INFO: `/v3/projects/df6ea6f7bee542f3b7edc57c00ec2e91` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:05.905778 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5908d52-4710-42fc-8ad4-32ee9137e4ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=df6ea6f7bee542f3b7edc57c00ec2e91)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:49:05.911549 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5908d52-4710-42fc-8ad4-32ee9137e4ac admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:49:06.065950 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-c5908d52-4710-42fc-8ad4-32ee9137e4ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df6ea6f7bee542f3b7edc57c00ec2e91', 'request_id': 'req-c5908d52-4710-42fc-8ad4-32ee9137e4ac'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:06.065950 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-c5908d52-4710-42fc-8ad4-32ee9137e4ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df6ea6f7bee542f3b7edc57c00ec2e91', 'request_id': 'req-c5908d52-4710-42fc-8ad4-32ee9137e4ac'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:06.300738 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-c5908d52-4710-42fc-8ad4-32ee9137e4ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df6ea6f7bee542f3b7edc57c00ec2e91', 'request_id': 'req-c5908d52-4710-42fc-8ad4-32ee9137e4ac'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:06.332587 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:49:06.337025 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-c5908d52-4710-42fc-8ad4-32ee9137e4ac admin admin] The token cache is being invalidate because project df6ea6f7bee542f3b7edc57c00ec2e91 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:49:06.337191 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-c5908d52-4710-42fc-8ad4-32ee9137e4ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:06.338023 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-c5908d52-4710-42fc-8ad4-32ee9137e4ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:06.339881 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1155/2300] 10.222.0.81 () {58 vars in 1232 bytes} [Wed Jul 20 22:49:05 2022] DELETE /identity/v3/projects/df6ea6f7bee542f3b7edc57c00ec2e91 => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:49:06.343782 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fe6156-cabd-430b-a2db-1e7502a4e448 service neutron] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:49:06.353953 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fe6156-cabd-430b-a2db-1e7502a4e448 service neutron] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:49:06.404827 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:49:06.419152 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6624a8-a6c8-46e3-91e2-c9c8fcbb5982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:49:06.420065 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6624a8-a6c8-46e3-91e2-c9c8fcbb5982 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:06.420344 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6624a8-a6c8-46e3-91e2-c9c8fcbb5982 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:06.420622 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6624a8-a6c8-46e3-91e2-c9c8fcbb5982 admin admin] PATH_INFO: `/v3/users/de86f19a0ee1427bbec6b186776cf963` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:06.421998 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6624a8-a6c8-46e3-91e2-c9c8fcbb5982 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de86f19a0ee1427bbec6b186776cf963)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:49:06.424008 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6624a8-a6c8-46e3-91e2-c9c8fcbb5982 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:49:06.440100 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114d1fbc-d354-4782-94d4-b1c85e99b81b None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:06.440433 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114d1fbc-d354-4782-94d4-b1c85e99b81b None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:06.440718 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114d1fbc-d354-4782-94d4-b1c85e99b81b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:06.484029 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-114d1fbc-d354-4782-94d4-b1c85e99b81b None None] MFA Rules not processed for user `1f237b9adae24d1ca2262d89eb000de4`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:49:06.499179 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-114d1fbc-d354-4782-94d4-b1c85e99b81b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:49:06.554022 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1156/2301] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:49:06 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:49:06.564898 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114d1fbc-d354-4782-94d4-b1c85e99b81b None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:49:06.566417 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-114d1fbc-d354-4782-94d4-b1c85e99b81b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:49:06.622387 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114d1fbc-d354-4782-94d4-b1c85e99b81b None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:49:06.627184 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92745688-9523-467b-90b5-bc6a281216cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecf4db289fba4e4a806198c62d31f7c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:49:06.630970 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92745688-9523-467b-90b5-bc6a281216cf service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:06.630970 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92745688-9523-467b-90b5-bc6a281216cf service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:06.630970 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92745688-9523-467b-90b5-bc6a281216cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:06.637426 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92745688-9523-467b-90b5-bc6a281216cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:49:06.638866 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92745688-9523-467b-90b5-bc6a281216cf service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:49:06.648447 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1157/2302] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:49:06 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 22:49:06.693878 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-6e6624a8-a6c8-46e3-91e2-c9c8fcbb5982 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de86f19a0ee1427bbec6b186776cf963'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:06.694627 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-6e6624a8-a6c8-46e3-91e2-c9c8fcbb5982 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de86f19a0ee1427bbec6b186776cf963'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:06.695340 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-6e6624a8-a6c8-46e3-91e2-c9c8fcbb5982 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de86f19a0ee1427bbec6b186776cf963'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:06.702689 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-6e6624a8-a6c8-46e3-91e2-c9c8fcbb5982 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de86f19a0ee1427bbec6b186776cf963'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:06.817728 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-6e6624a8-a6c8-46e3-91e2-c9c8fcbb5982 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de86f19a0ee1427bbec6b186776cf963'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:06.836240 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1146/2303] 10.222.0.81 () {58 vars in 1223 bytes} [Wed Jul 20 22:49:06 2022] DELETE /identity/v3/users/de86f19a0ee1427bbec6b186776cf963 => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:49:07.403026 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92745688-9523-467b-90b5-bc6a281216cf service cinder] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:49:07.405103 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-92745688-9523-467b-90b5-bc6a281216cf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:49:07.430969 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92745688-9523-467b-90b5-bc6a281216cf service cinder] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:49:07.486245 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df54eb5c-8dfb-42e1-93e0-8b837a1d86e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:49:07.487182 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df54eb5c-8dfb-42e1-93e0-8b837a1d86e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:07.487457 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df54eb5c-8dfb-42e1-93e0-8b837a1d86e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:07.487734 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df54eb5c-8dfb-42e1-93e0-8b837a1d86e3 admin admin] PATH_INFO: `/v3/projects/bf0e589e4c3946ea8e02bf7f08a7433b` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:07.489151 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df54eb5c-8dfb-42e1-93e0-8b837a1d86e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf0e589e4c3946ea8e02bf7f08a7433b)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:49:07.491285 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df54eb5c-8dfb-42e1-93e0-8b837a1d86e3 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:49:07.609239 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:49:07.619135 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-df54eb5c-8dfb-42e1-93e0-8b837a1d86e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf0e589e4c3946ea8e02bf7f08a7433b', 'request_id': 'req-df54eb5c-8dfb-42e1-93e0-8b837a1d86e3'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:07.628795 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-df54eb5c-8dfb-42e1-93e0-8b837a1d86e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf0e589e4c3946ea8e02bf7f08a7433b', 'request_id': 'req-df54eb5c-8dfb-42e1-93e0-8b837a1d86e3'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:07.702787 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-df54eb5c-8dfb-42e1-93e0-8b837a1d86e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf0e589e4c3946ea8e02bf7f08a7433b', 'request_id': 'req-df54eb5c-8dfb-42e1-93e0-8b837a1d86e3'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:07.714222 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-df54eb5c-8dfb-42e1-93e0-8b837a1d86e3 admin admin] The token cache is being invalidate because project bf0e589e4c3946ea8e02bf7f08a7433b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:49:07.714515 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-df54eb5c-8dfb-42e1-93e0-8b837a1d86e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:07.715179 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-df54eb5c-8dfb-42e1-93e0-8b837a1d86e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:07.716761 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1158/2304] 10.222.0.81 () {58 vars in 1232 bytes} [Wed Jul 20 22:49:07 2022] DELETE /identity/v3/projects/bf0e589e4c3946ea8e02bf7f08a7433b => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:49:07.722813 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f6104d-3276-4bc0-871e-2efb318a13bf None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:07.723323 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f6104d-3276-4bc0-871e-2efb318a13bf None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:07.723682 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f6104d-3276-4bc0-871e-2efb318a13bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:07.741635 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-a7f6104d-3276-4bc0-871e-2efb318a13bf None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:49:07.742827 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-a7f6104d-3276-4bc0-871e-2efb318a13bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:49:07.803544 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1147/2305] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:49:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:49:07.811221 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da767be-7e00-4700-bac2-c3f94a28a0e4 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:07.811511 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da767be-7e00-4700-bac2-c3f94a28a0e4 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:07.811789 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da767be-7e00-4700-bac2-c3f94a28a0e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:07.833889 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-8da767be-7e00-4700-bac2-c3f94a28a0e4 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:49:07.835148 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-8da767be-7e00-4700-bac2-c3f94a28a0e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:49:07.843493 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1159/2306] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:49:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:49:07.847894 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f6104d-3276-4bc0-871e-2efb318a13bf None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:49:07.859091 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f6104d-3276-4bc0-871e-2efb318a13bf None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:49:07.863786 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfffac7c-ff77-4a0d-b86e-f6b8c9c1b9f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:49:07.866100 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfffac7c-ff77-4a0d-b86e-f6b8c9c1b9f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:07.866100 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfffac7c-ff77-4a0d-b86e-f6b8c9c1b9f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:07.866100 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfffac7c-ff77-4a0d-b86e-f6b8c9c1b9f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:07.866100 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfffac7c-ff77-4a0d-b86e-f6b8c9c1b9f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:49:07.866100 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfffac7c-ff77-4a0d-b86e-f6b8c9c1b9f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:49:07.867685 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfffac7c-ff77-4a0d-b86e-f6b8c9c1b9f4 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:49:07.879967 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1148/2307] 10.222.0.81 () {58 vars in 1152 bytes} [Wed Jul 20 22:49:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:49:07.888305 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da767be-7e00-4700-bac2-c3f94a28a0e4 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:49:07.902779 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da767be-7e00-4700-bac2-c3f94a28a0e4 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:49:07.907628 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be91f70-6e34-4303-8a5e-769a640b7e83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:49:07.908295 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be91f70-6e34-4303-8a5e-769a640b7e83 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:07.908582 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be91f70-6e34-4303-8a5e-769a640b7e83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:07.909135 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be91f70-6e34-4303-8a5e-769a640b7e83 admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:07.909957 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be91f70-6e34-4303-8a5e-769a640b7e83 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:49:07.911954 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be91f70-6e34-4303-8a5e-769a640b7e83 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:49:08.025004 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1160/2308] 10.222.0.81 () {60 vars in 1152 bytes} [Wed Jul 20 22:49:07 2022] POST /identity/v3/projects => generated 383 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:49:08.029417 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfffac7c-ff77-4a0d-b86e-f6b8c9c1b9f4 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:49:08.039449 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfffac7c-ff77-4a0d-b86e-f6b8c9c1b9f4 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:49:08.088345 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:49:08.097100 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef4d665-d409-49a5-a892-9dc45aa9024e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:49:08.101977 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef4d665-d409-49a5-a892-9dc45aa9024e admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:08.101977 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef4d665-d409-49a5-a892-9dc45aa9024e admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:08.101977 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef4d665-d409-49a5-a892-9dc45aa9024e admin admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:08.101977 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef4d665-d409-49a5-a892-9dc45aa9024e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:49:08.101977 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef4d665-d409-49a5-a892-9dc45aa9024e admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:49:08.205546 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1149/2309] 10.222.0.81 () {60 vars in 1143 bytes} [Wed Jul 20 22:49:08 2022] POST /identity/v3/users => generated 292 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:49:08.209676 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be91f70-6e34-4303-8a5e-769a640b7e83 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:49:08.219900 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be91f70-6e34-4303-8a5e-769a640b7e83 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:49:08.224613 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd37e319-9324-44af-84a9-46c439acaff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:49:08.225282 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd37e319-9324-44af-84a9-46c439acaff4 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:08.225574 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd37e319-9324-44af-84a9-46c439acaff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:08.225858 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd37e319-9324-44af-84a9-46c439acaff4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:08.226634 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd37e319-9324-44af-84a9-46c439acaff4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:49:08.228234 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd37e319-9324-44af-84a9-46c439acaff4 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:49:08.239245 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1161/2310] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:49:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:49:08.243123 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef4d665-d409-49a5-a892-9dc45aa9024e admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:49:08.253529 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef4d665-d409-49a5-a892-9dc45aa9024e admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:49:08.258160 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4046d2ee-cb9c-4521-8865-914271a12868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:49:08.258808 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4046d2ee-cb9c-4521-8865-914271a12868 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:08.259131 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4046d2ee-cb9c-4521-8865-914271a12868 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:08.259407 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4046d2ee-cb9c-4521-8865-914271a12868 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:08.260121 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4046d2ee-cb9c-4521-8865-914271a12868 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:49:08.261719 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4046d2ee-cb9c-4521-8865-914271a12868 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:49:08.271703 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1150/2311] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:49:08 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:49:08.276000 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd37e319-9324-44af-84a9-46c439acaff4 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:49:08.287047 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd37e319-9324-44af-84a9-46c439acaff4 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:49:08.291829 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb56bf3-55f9-49c2-8182-6338fda8bbb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:49:08.292724 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb56bf3-55f9-49c2-8182-6338fda8bbb2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:08.293037 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb56bf3-55f9-49c2-8182-6338fda8bbb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:08.293321 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb56bf3-55f9-49c2-8182-6338fda8bbb2 admin admin] PATH_INFO: `/v3/projects/bc3873430c1241659f85113170b17d37/users/f76b22f6766344f3806d41e12506106c/roles/559942c4de1744169f8b5e0404ecde6b` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:08.308476 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb56bf3-55f9-49c2-8182-6338fda8bbb2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc3873430c1241659f85113170b17d37, user_id=f76b22f6766344f3806d41e12506106c, role_id=559942c4de1744169f8b5e0404ecde6b)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:49:08.311162 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb56bf3-55f9-49c2-8182-6338fda8bbb2 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:49:08.440051 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1162/2312] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:49:08 2022] PUT /identity/v3/projects/bc3873430c1241659f85113170b17d37/users/f76b22f6766344f3806d41e12506106c/roles/559942c4de1744169f8b5e0404ecde6b => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:49:08.451794 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428d2ca7-8723-4ded-bef8-4d5ea9514056 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:08.452721 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428d2ca7-8723-4ded-bef8-4d5ea9514056 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:08.453022 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428d2ca7-8723-4ded-bef8-4d5ea9514056 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:08.483902 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-428d2ca7-8723-4ded-bef8-4d5ea9514056 None None] MFA Rules not processed for user `f76b22f6766344f3806d41e12506106c`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:49:08.483902 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-428d2ca7-8723-4ded-bef8-4d5ea9514056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:49:08.550753 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1151/2313] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:49:08 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:49:08.554809 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb56bf3-55f9-49c2-8182-6338fda8bbb2 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:49:08.619002 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb56bf3-55f9-49c2-8182-6338fda8bbb2 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:49:08.623824 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3e6db0-27ba-40f5-b8ec-f729917e9466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:49:08.624492 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3e6db0-27ba-40f5-b8ec-f729917e9466 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:08.624774 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3e6db0-27ba-40f5-b8ec-f729917e9466 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:08.625054 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3e6db0-27ba-40f5-b8ec-f729917e9466 admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:08.625752 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3e6db0-27ba-40f5-b8ec-f729917e9466 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:49:08.627687 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3e6db0-27ba-40f5-b8ec-f729917e9466 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:49:08.722686 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1163/2314] 10.222.0.81 () {60 vars in 1152 bytes} [Wed Jul 20 22:49:08 2022] POST /identity/v3/projects => generated 381 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:49:08.726720 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428d2ca7-8723-4ded-bef8-4d5ea9514056 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:49:08.738220 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428d2ca7-8723-4ded-bef8-4d5ea9514056 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:49:08.793993 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc393b92-26c8-423d-8c4f-878f047288d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:49:08.794673 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc393b92-26c8-423d-8c4f-878f047288d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:08.794964 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc393b92-26c8-423d-8c4f-878f047288d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:08.795280 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc393b92-26c8-423d-8c4f-878f047288d6 admin admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:08.795980 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc393b92-26c8-423d-8c4f-878f047288d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:49:08.797964 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc393b92-26c8-423d-8c4f-878f047288d6 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:49:08.906295 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1152/2315] 10.222.0.81 () {60 vars in 1143 bytes} [Wed Jul 20 22:49:08 2022] POST /identity/v3/users => generated 297 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 22:49:08.913440 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3e6db0-27ba-40f5-b8ec-f729917e9466 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:49:08.924501 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3e6db0-27ba-40f5-b8ec-f729917e9466 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:49:08.930859 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc89db6-6aec-427c-9c7f-f17baae361f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:49:08.931594 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc89db6-6aec-427c-9c7f-f17baae361f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:08.931887 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc89db6-6aec-427c-9c7f-f17baae361f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:08.932172 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc89db6-6aec-427c-9c7f-f17baae361f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:08.932906 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc89db6-6aec-427c-9c7f-f17baae361f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:49:08.934874 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc89db6-6aec-427c-9c7f-f17baae361f3 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:49:08.950538 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1164/2316] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:49:08 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:49:08.951929 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc393b92-26c8-423d-8c4f-878f047288d6 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:49:08.962432 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc393b92-26c8-423d-8c4f-878f047288d6 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:49:08.968342 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c17810-436b-41e3-a9b4-9a6ade059a46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:49:08.968966 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c17810-436b-41e3-a9b4-9a6ade059a46 admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:08.969682 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c17810-436b-41e3-a9b4-9a6ade059a46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:08.969963 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c17810-436b-41e3-a9b4-9a6ade059a46 admin admin] PATH_INFO: `/v3/projects/f8fee54729ee45cb9f4011c725a9e271/users/46c1d04c57d34e8084207335fbe2629c/roles/d89b6c92d35a401a8517f1d47f6be843` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:08.991506 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c17810-436b-41e3-a9b4-9a6ade059a46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8fee54729ee45cb9f4011c725a9e271, user_id=46c1d04c57d34e8084207335fbe2629c, role_id=d89b6c92d35a401a8517f1d47f6be843)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:49:08.991506 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c17810-436b-41e3-a9b4-9a6ade059a46 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:49:09.119979 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1153/2317] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:49:08 2022] PUT /identity/v3/projects/f8fee54729ee45cb9f4011c725a9e271/users/46c1d04c57d34e8084207335fbe2629c/roles/d89b6c92d35a401a8517f1d47f6be843 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:49:09.123993 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc89db6-6aec-427c-9c7f-f17baae361f3 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:49:09.133914 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc89db6-6aec-427c-9c7f-f17baae361f3 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:49:09.165149 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:49:09.190709 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef35c4b-4a12-4300-b100-89574944d51a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:49:09.196399 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef35c4b-4a12-4300-b100-89574944d51a admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:09.196399 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef35c4b-4a12-4300-b100-89574944d51a admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:09.196399 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef35c4b-4a12-4300-b100-89574944d51a admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:09.196399 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef35c4b-4a12-4300-b100-89574944d51a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:49:09.196399 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef35c4b-4a12-4300-b100-89574944d51a admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:49:09.204963 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1165/2318] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:49:09 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:49:09.209440 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c17810-436b-41e3-a9b4-9a6ade059a46 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:49:09.220057 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c17810-436b-41e3-a9b4-9a6ade059a46 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:49:09.225430 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7de237e-6bf9-40f8-90d9-75cdb897586a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:49:09.226321 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7de237e-6bf9-40f8-90d9-75cdb897586a admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:09.226741 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7de237e-6bf9-40f8-90d9-75cdb897586a admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:09.227120 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7de237e-6bf9-40f8-90d9-75cdb897586a admin admin] PATH_INFO: `/v3/projects/f8fee54729ee45cb9f4011c725a9e271/users/46c1d04c57d34e8084207335fbe2629c/roles/d89b6c92d35a401a8517f1d47f6be843` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:09.229372 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7de237e-6bf9-40f8-90d9-75cdb897586a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8fee54729ee45cb9f4011c725a9e271, user_id=46c1d04c57d34e8084207335fbe2629c, role_id=d89b6c92d35a401a8517f1d47f6be843)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:49:09.232135 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7de237e-6bf9-40f8-90d9-75cdb897586a admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:49:09.242416 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1154/2319] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:49:09 2022] PUT /identity/v3/projects/f8fee54729ee45cb9f4011c725a9e271/users/46c1d04c57d34e8084207335fbe2629c/roles/d89b6c92d35a401a8517f1d47f6be843 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:49:09.255577 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7075385-25aa-431b-88aa-3fea502bb8ae None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:09.255958 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7075385-25aa-431b-88aa-3fea502bb8ae None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:09.256284 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7075385-25aa-431b-88aa-3fea502bb8ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:09.283510 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-d7075385-25aa-431b-88aa-3fea502bb8ae None None] MFA Rules not processed for user `46c1d04c57d34e8084207335fbe2629c`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:49:09.284916 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-d7075385-25aa-431b-88aa-3fea502bb8ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:49:09.356153 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1166/2320] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:49:09 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:49:09.368171 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7de237e-6bf9-40f8-90d9-75cdb897586a admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:49:09.369768 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-e7de237e-6bf9-40f8-90d9-75cdb897586a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:49:09.385592 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:49:09.428490 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7de237e-6bf9-40f8-90d9-75cdb897586a admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:49:09.432207 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a8db05-76be-4b11-8473-6df519b13353 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecf4db289fba4e4a806198c62d31f7c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:49:09.432853 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a8db05-76be-4b11-8473-6df519b13353 service cinder] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:09.438426 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a8db05-76be-4b11-8473-6df519b13353 service cinder] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:09.438426 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a8db05-76be-4b11-8473-6df519b13353 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:09.443340 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a8db05-76be-4b11-8473-6df519b13353 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:49:09.444857 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a8db05-76be-4b11-8473-6df519b13353 service cinder] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:49:09.454513 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1155/2321] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:49:09 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:49:09.636265 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7075385-25aa-431b-88aa-3fea502bb8ae None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:49:09.644659 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a554a0c9-8aea-40f2-9d45-0de4847902ad None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:09.645026 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a554a0c9-8aea-40f2-9d45-0de4847902ad None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:09.645380 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a554a0c9-8aea-40f2-9d45-0de4847902ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:09.646678 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7075385-25aa-431b-88aa-3fea502bb8ae None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:49:09.650487 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8a6f9e-9cce-445b-930b-b47cee3cfcb0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecf4db289fba4e4a806198c62d31f7c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:49:09.651173 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8a6f9e-9cce-445b-930b-b47cee3cfcb0 service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:09.651718 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8a6f9e-9cce-445b-930b-b47cee3cfcb0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:09.651995 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8a6f9e-9cce-445b-930b-b47cee3cfcb0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:09.666658 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-a554a0c9-8aea-40f2-9d45-0de4847902ad None None] MFA Rules not processed for user `1f237b9adae24d1ca2262d89eb000de4`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:49:09.667911 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-a554a0c9-8aea-40f2-9d45-0de4847902ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:49:09.715023 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8a6f9e-9cce-445b-930b-b47cee3cfcb0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:49:09.716518 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8a6f9e-9cce-445b-930b-b47cee3cfcb0 service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:49:09.724320 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1156/2322] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:49:09 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 22:49:09.726245 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1167/2323] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:49:09 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:49:10.333990 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a554a0c9-8aea-40f2-9d45-0de4847902ad None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:49:10.398151 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a554a0c9-8aea-40f2-9d45-0de4847902ad None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:49:10.401032 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a79d6f-3045-4092-866a-b73a55649916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:49:10.402177 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a79d6f-3045-4092-866a-b73a55649916 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:10.402458 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a79d6f-3045-4092-866a-b73a55649916 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:10.402741 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a79d6f-3045-4092-866a-b73a55649916 admin admin] PATH_INFO: `/v3/users/f76b22f6766344f3806d41e12506106c` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:10.404033 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a79d6f-3045-4092-866a-b73a55649916 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f76b22f6766344f3806d41e12506106c)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:49:10.405950 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a79d6f-3045-4092-866a-b73a55649916 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:49:10.693543 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-28a79d6f-3045-4092-866a-b73a55649916 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f76b22f6766344f3806d41e12506106c'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:10.694210 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-28a79d6f-3045-4092-866a-b73a55649916 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f76b22f6766344f3806d41e12506106c'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:10.694868 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-28a79d6f-3045-4092-866a-b73a55649916 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f76b22f6766344f3806d41e12506106c'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:10.702391 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-28a79d6f-3045-4092-866a-b73a55649916 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f76b22f6766344f3806d41e12506106c'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:10.804855 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-28a79d6f-3045-4092-866a-b73a55649916 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f76b22f6766344f3806d41e12506106c'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:10.806217 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:49:10.824009 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1157/2324] 10.222.0.81 () {58 vars in 1223 bytes} [Wed Jul 20 22:49:10 2022] DELETE /identity/v3/users/f76b22f6766344f3806d41e12506106c => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:49:10.828582 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6c6d42-b449-43b5-a588-34a48f0673ff None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:10.829043 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6c6d42-b449-43b5-a588-34a48f0673ff None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:10.829137 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6c6d42-b449-43b5-a588-34a48f0673ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:10.846531 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-fb6c6d42-b449-43b5-a588-34a48f0673ff None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:49:10.847823 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-fb6c6d42-b449-43b5-a588-34a48f0673ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:49:10.894602 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:49:10.907144 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1168/2325] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:49:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 22:49:10.916513 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a79d6f-3045-4092-866a-b73a55649916 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:49:10.918055 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-28a79d6f-3045-4092-866a-b73a55649916 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:49:10.976001 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a79d6f-3045-4092-866a-b73a55649916 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:49:10.979580 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46f77e0-e06c-4e17-a3de-031f0fd80987 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:49:10.980228 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46f77e0-e06c-4e17-a3de-031f0fd80987 service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:10.980513 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46f77e0-e06c-4e17-a3de-031f0fd80987 service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:10.980788 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46f77e0-e06c-4e17-a3de-031f0fd80987 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:10.989782 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46f77e0-e06c-4e17-a3de-031f0fd80987 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:49:10.991391 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46f77e0-e06c-4e17-a3de-031f0fd80987 service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:49:11.001917 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1158/2326] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:49:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:49:11.547771 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6c6d42-b449-43b5-a588-34a48f0673ff None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:49:11.547771 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-fb6c6d42-b449-43b5-a588-34a48f0673ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:49:11.559645 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6c6d42-b449-43b5-a588-34a48f0673ff None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:49:11.564508 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1ef8a8-8355-4c68-8eca-77e071d33829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:49:11.565900 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1ef8a8-8355-4c68-8eca-77e071d33829 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:11.566188 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1ef8a8-8355-4c68-8eca-77e071d33829 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:11.566470 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1ef8a8-8355-4c68-8eca-77e071d33829 admin admin] PATH_INFO: `/v3/projects/bc3873430c1241659f85113170b17d37` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:11.567966 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1ef8a8-8355-4c68-8eca-77e071d33829 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc3873430c1241659f85113170b17d37)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:49:11.570098 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1ef8a8-8355-4c68-8eca-77e071d33829 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:49:11.702061 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-8f1ef8a8-8355-4c68-8eca-77e071d33829 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc3873430c1241659f85113170b17d37', 'request_id': 'req-8f1ef8a8-8355-4c68-8eca-77e071d33829'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:11.710398 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-8f1ef8a8-8355-4c68-8eca-77e071d33829 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc3873430c1241659f85113170b17d37', 'request_id': 'req-8f1ef8a8-8355-4c68-8eca-77e071d33829'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:11.805007 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-8f1ef8a8-8355-4c68-8eca-77e071d33829 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc3873430c1241659f85113170b17d37', 'request_id': 'req-8f1ef8a8-8355-4c68-8eca-77e071d33829'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:11.816893 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-8f1ef8a8-8355-4c68-8eca-77e071d33829 admin admin] The token cache is being invalidate because project bc3873430c1241659f85113170b17d37 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:49:11.817180 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-8f1ef8a8-8355-4c68-8eca-77e071d33829 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:11.820226 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-8f1ef8a8-8355-4c68-8eca-77e071d33829 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:11.820226 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1169/2327] 10.222.0.81 () {58 vars in 1232 bytes} [Wed Jul 20 22:49:11 2022] DELETE /identity/v3/projects/bc3873430c1241659f85113170b17d37 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:49:11.823439 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46f77e0-e06c-4e17-a3de-031f0fd80987 service neutron] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:49:11.834238 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46f77e0-e06c-4e17-a3de-031f0fd80987 service neutron] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:49:11.890706 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa4b056-bffa-401c-9828-038a2b27211f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:49:11.891621 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa4b056-bffa-401c-9828-038a2b27211f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:11.891912 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa4b056-bffa-401c-9828-038a2b27211f admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:11.892198 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa4b056-bffa-401c-9828-038a2b27211f admin admin] PATH_INFO: `/v3/users/46c1d04c57d34e8084207335fbe2629c` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:11.893422 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa4b056-bffa-401c-9828-038a2b27211f admin admin] RBAC: Authorizing `identity:delete_user(user_id=46c1d04c57d34e8084207335fbe2629c)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:49:11.895453 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa4b056-bffa-401c-9828-038a2b27211f admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:49:12.127129 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-7aa4b056-bffa-401c-9828-038a2b27211f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46c1d04c57d34e8084207335fbe2629c'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:12.128338 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-7aa4b056-bffa-401c-9828-038a2b27211f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46c1d04c57d34e8084207335fbe2629c'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:12.128947 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-7aa4b056-bffa-401c-9828-038a2b27211f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46c1d04c57d34e8084207335fbe2629c'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:12.135889 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-7aa4b056-bffa-401c-9828-038a2b27211f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46c1d04c57d34e8084207335fbe2629c'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:12.228211 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-7aa4b056-bffa-401c-9828-038a2b27211f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46c1d04c57d34e8084207335fbe2629c'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:12.266376 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1159/2328] 10.222.0.81 () {58 vars in 1223 bytes} [Wed Jul 20 22:49:11 2022] DELETE /identity/v3/users/46c1d04c57d34e8084207335fbe2629c => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:49:12.728548 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1ef8a8-8355-4c68-8eca-77e071d33829 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:49:12.730128 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-8f1ef8a8-8355-4c68-8eca-77e071d33829 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:49:12.744149 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1ef8a8-8355-4c68-8eca-77e071d33829 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:49:12.804966 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5027eb4-cc75-4012-bf59-08ef5a1be03b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:49:12.805900 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5027eb4-cc75-4012-bf59-08ef5a1be03b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:12.806195 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5027eb4-cc75-4012-bf59-08ef5a1be03b admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:12.806477 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5027eb4-cc75-4012-bf59-08ef5a1be03b admin admin] PATH_INFO: `/v3/projects/f8fee54729ee45cb9f4011c725a9e271` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:12.807748 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5027eb4-cc75-4012-bf59-08ef5a1be03b admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8fee54729ee45cb9f4011c725a9e271)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:49:12.809707 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5027eb4-cc75-4012-bf59-08ef5a1be03b admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:49:12.922163 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-f5027eb4-cc75-4012-bf59-08ef5a1be03b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8fee54729ee45cb9f4011c725a9e271', 'request_id': 'req-f5027eb4-cc75-4012-bf59-08ef5a1be03b'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:12.928916 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-f5027eb4-cc75-4012-bf59-08ef5a1be03b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8fee54729ee45cb9f4011c725a9e271', 'request_id': 'req-f5027eb4-cc75-4012-bf59-08ef5a1be03b'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:12.999506 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-f5027eb4-cc75-4012-bf59-08ef5a1be03b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8fee54729ee45cb9f4011c725a9e271', 'request_id': 'req-f5027eb4-cc75-4012-bf59-08ef5a1be03b'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:13.013096 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-f5027eb4-cc75-4012-bf59-08ef5a1be03b admin admin] The token cache is being invalidate because project f8fee54729ee45cb9f4011c725a9e271 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:49:13.013396 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-f5027eb4-cc75-4012-bf59-08ef5a1be03b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:13.014068 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-f5027eb4-cc75-4012-bf59-08ef5a1be03b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:13.015670 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1170/2329] 10.222.0.81 () {58 vars in 1232 bytes} [Wed Jul 20 22:49:12 2022] DELETE /identity/v3/projects/f8fee54729ee45cb9f4011c725a9e271 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:49:13.021299 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcb5756-f918-4222-9416-61efd54dcd85 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:13.021762 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcb5756-f918-4222-9416-61efd54dcd85 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:13.021962 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcb5756-f918-4222-9416-61efd54dcd85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:13.028600 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:49:13.040608 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-8fcb5756-f918-4222-9416-61efd54dcd85 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:49:13.041840 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-8fcb5756-f918-4222-9416-61efd54dcd85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:49:13.099944 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1160/2330] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:49:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 22:49:13.107987 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02be1ebc-c06d-440d-b251-0ad6b49a9643 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:13.108240 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02be1ebc-c06d-440d-b251-0ad6b49a9643 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:13.113782 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02be1ebc-c06d-440d-b251-0ad6b49a9643 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:13.120701 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:49:13.126624 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-02be1ebc-c06d-440d-b251-0ad6b49a9643 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:49:13.128358 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-02be1ebc-c06d-440d-b251-0ad6b49a9643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:49:13.136501 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1171/2331] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:49:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:49:13.141694 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcb5756-f918-4222-9416-61efd54dcd85 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:49:13.157180 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcb5756-f918-4222-9416-61efd54dcd85 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:49:13.162093 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d11ee85-7a80-489f-a0f7-641843f0f631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:49:13.162766 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d11ee85-7a80-489f-a0f7-641843f0f631 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:13.163125 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d11ee85-7a80-489f-a0f7-641843f0f631 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:13.163418 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d11ee85-7a80-489f-a0f7-641843f0f631 admin admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:13.164398 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d11ee85-7a80-489f-a0f7-641843f0f631 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:49:13.164893 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d11ee85-7a80-489f-a0f7-641843f0f631 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:49:13.166671 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d11ee85-7a80-489f-a0f7-641843f0f631 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:49:13.179700 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1161/2332] 10.222.0.81 () {58 vars in 1152 bytes} [Wed Jul 20 22:49:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:49:13.183479 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02be1ebc-c06d-440d-b251-0ad6b49a9643 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:49:13.193844 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02be1ebc-c06d-440d-b251-0ad6b49a9643 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:49:13.198738 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddcce19-de79-4f43-b1d3-b02557b4497a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:49:13.199463 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddcce19-de79-4f43-b1d3-b02557b4497a admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:13.199751 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddcce19-de79-4f43-b1d3-b02557b4497a admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:13.200031 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddcce19-de79-4f43-b1d3-b02557b4497a admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:13.200719 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddcce19-de79-4f43-b1d3-b02557b4497a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:49:13.202620 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddcce19-de79-4f43-b1d3-b02557b4497a admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:49:13.303474 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1172/2333] 10.222.0.81 () {60 vars in 1152 bytes} [Wed Jul 20 22:49:13 2022] POST /identity/v3/projects => generated 383 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:49:13.307594 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d11ee85-7a80-489f-a0f7-641843f0f631 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:49:13.317980 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d11ee85-7a80-489f-a0f7-641843f0f631 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:49:13.373847 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d4d088-0077-4303-9b8c-2a36a71d54b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:49:13.374524 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d4d088-0077-4303-9b8c-2a36a71d54b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:13.374812 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d4d088-0077-4303-9b8c-2a36a71d54b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:13.375129 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d4d088-0077-4303-9b8c-2a36a71d54b6 admin admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:13.375827 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d4d088-0077-4303-9b8c-2a36a71d54b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:49:13.378331 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d4d088-0077-4303-9b8c-2a36a71d54b6 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:49:13.503833 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1162/2334] 10.222.0.81 () {60 vars in 1143 bytes} [Wed Jul 20 22:49:13 2022] POST /identity/v3/users => generated 292 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:49:13.508159 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddcce19-de79-4f43-b1d3-b02557b4497a admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:49:13.518966 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddcce19-de79-4f43-b1d3-b02557b4497a admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:49:13.523574 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48809221-4071-43f5-a2b4-e573c6ee382e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:49:13.524247 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48809221-4071-43f5-a2b4-e573c6ee382e admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:13.524536 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48809221-4071-43f5-a2b4-e573c6ee382e admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:13.524818 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48809221-4071-43f5-a2b4-e573c6ee382e admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:13.525548 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48809221-4071-43f5-a2b4-e573c6ee382e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:49:13.527439 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48809221-4071-43f5-a2b4-e573c6ee382e admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:49:13.537815 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1173/2335] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:49:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 22:49:13.541732 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d4d088-0077-4303-9b8c-2a36a71d54b6 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:49:13.552388 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d4d088-0077-4303-9b8c-2a36a71d54b6 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:49:13.556946 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb6b08a-2485-49ce-a281-c250f18ecd6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:49:13.557602 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb6b08a-2485-49ce-a281-c250f18ecd6c admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:13.557885 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb6b08a-2485-49ce-a281-c250f18ecd6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:13.558264 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb6b08a-2485-49ce-a281-c250f18ecd6c admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:13.558984 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb6b08a-2485-49ce-a281-c250f18ecd6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:49:13.560662 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb6b08a-2485-49ce-a281-c250f18ecd6c admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:49:13.571233 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1163/2336] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:49:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:49:13.575261 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48809221-4071-43f5-a2b4-e573c6ee382e admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:49:13.585246 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48809221-4071-43f5-a2b4-e573c6ee382e admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:49:13.589738 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd4743d-cad8-4039-9790-3d78197102d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:49:13.590665 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd4743d-cad8-4039-9790-3d78197102d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:13.590951 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd4743d-cad8-4039-9790-3d78197102d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:13.591265 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd4743d-cad8-4039-9790-3d78197102d1 admin admin] PATH_INFO: `/v3/projects/57fed9e4ecfe40e7a135c45436b207b3/users/70c89385badc4f5eab13108ca85a2de4/roles/559942c4de1744169f8b5e0404ecde6b` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:13.605521 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd4743d-cad8-4039-9790-3d78197102d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57fed9e4ecfe40e7a135c45436b207b3, user_id=70c89385badc4f5eab13108ca85a2de4, role_id=559942c4de1744169f8b5e0404ecde6b)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:49:13.608432 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd4743d-cad8-4039-9790-3d78197102d1 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:49:13.691321 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1174/2337] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:49:13 2022] PUT /identity/v3/projects/57fed9e4ecfe40e7a135c45436b207b3/users/70c89385badc4f5eab13108ca85a2de4/roles/559942c4de1744169f8b5e0404ecde6b => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:49:13.706683 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab74249-12c3-4913-9a4d-51742be16dfc None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:13.706986 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab74249-12c3-4913-9a4d-51742be16dfc None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:13.707370 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab74249-12c3-4913-9a4d-51742be16dfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:13.734047 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-4ab74249-12c3-4913-9a4d-51742be16dfc None None] MFA Rules not processed for user `70c89385badc4f5eab13108ca85a2de4`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:49:13.735350 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-4ab74249-12c3-4913-9a4d-51742be16dfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:49:13.797893 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:49:13.804588 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1164/2338] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:49:13 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:49:13.809397 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd4743d-cad8-4039-9790-3d78197102d1 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:49:13.874414 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd4743d-cad8-4039-9790-3d78197102d1 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:49:13.879413 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eb8707-c955-4769-a3bd-3a9e3ba8416a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:49:13.880143 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eb8707-c955-4769-a3bd-3a9e3ba8416a admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:13.880430 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eb8707-c955-4769-a3bd-3a9e3ba8416a admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:13.880741 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eb8707-c955-4769-a3bd-3a9e3ba8416a admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:13.881575 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65eb8707-c955-4769-a3bd-3a9e3ba8416a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:49:13.883546 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65eb8707-c955-4769-a3bd-3a9e3ba8416a admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:49:13.960401 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1175/2339] 10.222.0.81 () {60 vars in 1152 bytes} [Wed Jul 20 22:49:13 2022] POST /identity/v3/projects => generated 383 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 22:49:13.964577 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab74249-12c3-4913-9a4d-51742be16dfc None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:49:13.976729 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab74249-12c3-4913-9a4d-51742be16dfc None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:49:14.041282 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b048285-9cb2-4e5d-9ee4-785a8a5143b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:49:14.041978 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b048285-9cb2-4e5d-9ee4-785a8a5143b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:14.042292 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b048285-9cb2-4e5d-9ee4-785a8a5143b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:14.042553 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b048285-9cb2-4e5d-9ee4-785a8a5143b3 admin admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:14.043633 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b048285-9cb2-4e5d-9ee4-785a8a5143b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:49:14.045402 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b048285-9cb2-4e5d-9ee4-785a8a5143b3 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:49:14.136651 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1165/2340] 10.222.0.81 () {60 vars in 1143 bytes} [Wed Jul 20 22:49:13 2022] POST /identity/v3/users => generated 298 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:49:14.150989 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eb8707-c955-4769-a3bd-3a9e3ba8416a admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:49:14.162683 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eb8707-c955-4769-a3bd-3a9e3ba8416a admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:49:14.169128 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391f3d7b-2026-4d95-bc75-7e8a78f46aef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:49:14.169839 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391f3d7b-2026-4d95-bc75-7e8a78f46aef admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:14.183358 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391f3d7b-2026-4d95-bc75-7e8a78f46aef admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:14.183358 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391f3d7b-2026-4d95-bc75-7e8a78f46aef admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:14.183358 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391f3d7b-2026-4d95-bc75-7e8a78f46aef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:49:14.183358 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391f3d7b-2026-4d95-bc75-7e8a78f46aef admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:49:14.189321 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1176/2341] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:49:14 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:49:14.195739 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b048285-9cb2-4e5d-9ee4-785a8a5143b3 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:49:14.206564 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b048285-9cb2-4e5d-9ee4-785a8a5143b3 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:49:14.211421 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d851b81-3857-415f-b9bc-e43bd4340ea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:49:14.212314 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d851b81-3857-415f-b9bc-e43bd4340ea5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:14.212599 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d851b81-3857-415f-b9bc-e43bd4340ea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:14.212877 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d851b81-3857-415f-b9bc-e43bd4340ea5 admin admin] PATH_INFO: `/v3/projects/c6bb7979fa9b4fb59f8affa69b9f52c8/users/fee70ddfc8f640dca4875ac178b7344d/roles/d89b6c92d35a401a8517f1d47f6be843` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:14.228206 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d851b81-3857-415f-b9bc-e43bd4340ea5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6bb7979fa9b4fb59f8affa69b9f52c8, user_id=fee70ddfc8f640dca4875ac178b7344d, role_id=d89b6c92d35a401a8517f1d47f6be843)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:49:14.236102 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d851b81-3857-415f-b9bc-e43bd4340ea5 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:49:14.313957 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1166/2342] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:49:14 2022] PUT /identity/v3/projects/c6bb7979fa9b4fb59f8affa69b9f52c8/users/fee70ddfc8f640dca4875ac178b7344d/roles/d89b6c92d35a401a8517f1d47f6be843 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:49:14.318206 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391f3d7b-2026-4d95-bc75-7e8a78f46aef admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:49:14.329014 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391f3d7b-2026-4d95-bc75-7e8a78f46aef admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:49:14.385856 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955b60dd-fb0a-470a-aee2-7802226e8a6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:49:14.386536 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955b60dd-fb0a-470a-aee2-7802226e8a6c admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:14.386867 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955b60dd-fb0a-470a-aee2-7802226e8a6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:14.387309 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955b60dd-fb0a-470a-aee2-7802226e8a6c admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:14.388081 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955b60dd-fb0a-470a-aee2-7802226e8a6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:49:14.389668 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955b60dd-fb0a-470a-aee2-7802226e8a6c admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:49:14.399898 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1177/2343] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:49:14 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:49:14.404467 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d851b81-3857-415f-b9bc-e43bd4340ea5 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:49:14.415430 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d851b81-3857-415f-b9bc-e43bd4340ea5 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:49:14.420094 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1f5db8-3fb6-421d-9282-e8c53022af93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:49:14.421022 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1f5db8-3fb6-421d-9282-e8c53022af93 admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:14.421268 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1f5db8-3fb6-421d-9282-e8c53022af93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:14.421552 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1f5db8-3fb6-421d-9282-e8c53022af93 admin admin] PATH_INFO: `/v3/projects/c6bb7979fa9b4fb59f8affa69b9f52c8/users/fee70ddfc8f640dca4875ac178b7344d/roles/d89b6c92d35a401a8517f1d47f6be843` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:14.423561 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1f5db8-3fb6-421d-9282-e8c53022af93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6bb7979fa9b4fb59f8affa69b9f52c8, user_id=fee70ddfc8f640dca4875ac178b7344d, role_id=d89b6c92d35a401a8517f1d47f6be843)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:49:14.426592 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1f5db8-3fb6-421d-9282-e8c53022af93 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:49:14.437194 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1167/2344] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:49:14 2022] PUT /identity/v3/projects/c6bb7979fa9b4fb59f8affa69b9f52c8/users/fee70ddfc8f640dca4875ac178b7344d/roles/d89b6c92d35a401a8517f1d47f6be843 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:49:14.447670 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356fedf2-c474-4a37-9060-181be907dab8 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:14.447971 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356fedf2-c474-4a37-9060-181be907dab8 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:14.448254 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356fedf2-c474-4a37-9060-181be907dab8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:14.474334 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:49:14.477635 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-356fedf2-c474-4a37-9060-181be907dab8 None None] MFA Rules not processed for user `fee70ddfc8f640dca4875ac178b7344d`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:49:14.478895 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-356fedf2-c474-4a37-9060-181be907dab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:49:14.546992 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1178/2345] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:49:14 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:49:14.558132 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1f5db8-3fb6-421d-9282-e8c53022af93 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:49:14.559947 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-8b1f5db8-3fb6-421d-9282-e8c53022af93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:49:14.619714 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1f5db8-3fb6-421d-9282-e8c53022af93 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:49:14.623390 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2033ff6e-a8e4-4123-9a78-2f3f480b846b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecf4db289fba4e4a806198c62d31f7c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:49:14.624043 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2033ff6e-a8e4-4123-9a78-2f3f480b846b service cinder] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:14.624327 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2033ff6e-a8e4-4123-9a78-2f3f480b846b service cinder] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:14.624606 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2033ff6e-a8e4-4123-9a78-2f3f480b846b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:14.683312 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2033ff6e-a8e4-4123-9a78-2f3f480b846b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:49:14.684743 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2033ff6e-a8e4-4123-9a78-2f3f480b846b service cinder] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:49:14.695296 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1168/2346] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:49:14 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:49:17.937237 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356fedf2-c474-4a37-9060-181be907dab8 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:49:17.941465 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-356fedf2-c474-4a37-9060-181be907dab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:49:18.002006 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356fedf2-c474-4a37-9060-181be907dab8 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:49:18.006510 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32c7a9c-8b08-4081-a445-3a571b687518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:49:18.007419 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32c7a9c-8b08-4081-a445-3a571b687518 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:18.007701 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32c7a9c-8b08-4081-a445-3a571b687518 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:18.007974 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32c7a9c-8b08-4081-a445-3a571b687518 admin admin] PATH_INFO: `/v3/users/de57e5e148804737bf51412acc4d7cc0` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:18.009188 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32c7a9c-8b08-4081-a445-3a571b687518 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de57e5e148804737bf51412acc4d7cc0)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:49:18.011234 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32c7a9c-8b08-4081-a445-3a571b687518 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:49:18.149094 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-e32c7a9c-8b08-4081-a445-3a571b687518 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de57e5e148804737bf51412acc4d7cc0'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:18.149775 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-e32c7a9c-8b08-4081-a445-3a571b687518 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de57e5e148804737bf51412acc4d7cc0'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:18.150385 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-e32c7a9c-8b08-4081-a445-3a571b687518 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de57e5e148804737bf51412acc4d7cc0'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:18.157504 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-e32c7a9c-8b08-4081-a445-3a571b687518 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de57e5e148804737bf51412acc4d7cc0'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:18.227630 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-e32c7a9c-8b08-4081-a445-3a571b687518 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de57e5e148804737bf51412acc4d7cc0'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:18.245901 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1179/2347] 10.222.0.81 () {58 vars in 1223 bytes} [Wed Jul 20 22:49:17 2022] DELETE /identity/v3/users/de57e5e148804737bf51412acc4d7cc0 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:49:18.250471 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8cf499-38ae-4a6c-b502-52f8f80f18fe None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:18.250971 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8cf499-38ae-4a6c-b502-52f8f80f18fe None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:18.251110 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8cf499-38ae-4a6c-b502-52f8f80f18fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:18.257648 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:49:18.269409 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-9b8cf499-38ae-4a6c-b502-52f8f80f18fe None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:49:18.270670 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-9b8cf499-38ae-4a6c-b502-52f8f80f18fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:49:18.329784 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1169/2348] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:49:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:49:18.339588 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32c7a9c-8b08-4081-a445-3a571b687518 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:49:18.341061 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-e32c7a9c-8b08-4081-a445-3a571b687518 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:49:18.364551 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:49:18.402986 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32c7a9c-8b08-4081-a445-3a571b687518 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:49:18.406805 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ed945c-758d-497e-94f7-68fa3b137440 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:49:18.407566 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ed945c-758d-497e-94f7-68fa3b137440 service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:18.407846 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ed945c-758d-497e-94f7-68fa3b137440 service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:18.408122 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ed945c-758d-497e-94f7-68fa3b137440 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:18.418319 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ed945c-758d-497e-94f7-68fa3b137440 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:49:18.419804 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ed945c-758d-497e-94f7-68fa3b137440 service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:49:18.430342 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1180/2349] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:49:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:49:18.897224 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8cf499-38ae-4a6c-b502-52f8f80f18fe None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:49:18.898665 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-9b8cf499-38ae-4a6c-b502-52f8f80f18fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:49:18.911858 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8cf499-38ae-4a6c-b502-52f8f80f18fe None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:49:18.916710 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad96adb7-7dd2-4bd8-95bb-c8e620555cee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:49:18.917597 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad96adb7-7dd2-4bd8-95bb-c8e620555cee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:18.917937 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad96adb7-7dd2-4bd8-95bb-c8e620555cee admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:18.918163 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad96adb7-7dd2-4bd8-95bb-c8e620555cee admin admin] PATH_INFO: `/v3/projects/23671aa0492d49eb9749e5dc12643038` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:18.923640 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad96adb7-7dd2-4bd8-95bb-c8e620555cee admin admin] RBAC: Authorizing `identity:delete_project(project_id=23671aa0492d49eb9749e5dc12643038)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:49:18.923640 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad96adb7-7dd2-4bd8-95bb-c8e620555cee admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:49:19.029299 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-ad96adb7-7dd2-4bd8-95bb-c8e620555cee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23671aa0492d49eb9749e5dc12643038', 'request_id': 'req-ad96adb7-7dd2-4bd8-95bb-c8e620555cee'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:19.090278 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-ad96adb7-7dd2-4bd8-95bb-c8e620555cee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23671aa0492d49eb9749e5dc12643038', 'request_id': 'req-ad96adb7-7dd2-4bd8-95bb-c8e620555cee'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:19.096422 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-ad96adb7-7dd2-4bd8-95bb-c8e620555cee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23671aa0492d49eb9749e5dc12643038', 'request_id': 'req-ad96adb7-7dd2-4bd8-95bb-c8e620555cee'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:19.107490 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-ad96adb7-7dd2-4bd8-95bb-c8e620555cee admin admin] The token cache is being invalidate because project 23671aa0492d49eb9749e5dc12643038 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:49:19.107777 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-ad96adb7-7dd2-4bd8-95bb-c8e620555cee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:19.108400 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-ad96adb7-7dd2-4bd8-95bb-c8e620555cee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:19.112505 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1170/2350] 10.222.0.81 () {58 vars in 1232 bytes} [Wed Jul 20 22:49:18 2022] DELETE /identity/v3/projects/23671aa0492d49eb9749e5dc12643038 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:49:19.114966 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a681bc4-df5e-4443-8c81-836535b623a3 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:19.115458 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a681bc4-df5e-4443-8c81-836535b623a3 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:19.115626 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a681bc4-df5e-4443-8c81-836535b623a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:19.133786 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-8a681bc4-df5e-4443-8c81-836535b623a3 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:49:19.134995 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-8a681bc4-df5e-4443-8c81-836535b623a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:49:19.192937 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1181/2351] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:49:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:49:19.201264 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a74501-a91c-4299-9466-00e50b3e2658 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:19.201557 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a74501-a91c-4299-9466-00e50b3e2658 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:19.201841 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a74501-a91c-4299-9466-00e50b3e2658 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:19.226379 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-02a74501-a91c-4299-9466-00e50b3e2658 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:49:19.226379 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-02a74501-a91c-4299-9466-00e50b3e2658 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:49:19.229604 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1171/2352] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:49:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:49:19.235130 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a681bc4-df5e-4443-8c81-836535b623a3 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:49:19.246087 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a681bc4-df5e-4443-8c81-836535b623a3 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:49:19.251257 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec82161b-236b-46a4-81d7-cb13301fc6fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:49:19.252444 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec82161b-236b-46a4-81d7-cb13301fc6fa admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:19.252746 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec82161b-236b-46a4-81d7-cb13301fc6fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:19.253030 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec82161b-236b-46a4-81d7-cb13301fc6fa admin admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:19.253763 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec82161b-236b-46a4-81d7-cb13301fc6fa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:49:19.254074 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec82161b-236b-46a4-81d7-cb13301fc6fa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:49:19.255714 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec82161b-236b-46a4-81d7-cb13301fc6fa admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:49:19.268417 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1182/2353] 10.222.0.81 () {58 vars in 1152 bytes} [Wed Jul 20 22:49:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:49:19.272771 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a74501-a91c-4299-9466-00e50b3e2658 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:49:19.287030 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a74501-a91c-4299-9466-00e50b3e2658 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:49:19.292171 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345e2a3a-9f68-47ea-8d25-a2246079a598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:49:19.292829 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345e2a3a-9f68-47ea-8d25-a2246079a598 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:19.295357 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345e2a3a-9f68-47ea-8d25-a2246079a598 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:19.295357 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345e2a3a-9f68-47ea-8d25-a2246079a598 admin admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:19.295357 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345e2a3a-9f68-47ea-8d25-a2246079a598 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:49:19.295965 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345e2a3a-9f68-47ea-8d25-a2246079a598 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:49:19.435178 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1172/2354] 10.222.0.81 () {60 vars in 1152 bytes} [Wed Jul 20 22:49:19 2022] POST /identity/v3/projects => generated 397 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:49:19.438940 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec82161b-236b-46a4-81d7-cb13301fc6fa admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:49:19.449819 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec82161b-236b-46a4-81d7-cb13301fc6fa admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:49:19.505847 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa852d8-b172-4a7f-a50d-b46c074e0d79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:49:19.506556 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa852d8-b172-4a7f-a50d-b46c074e0d79 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:19.506844 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa852d8-b172-4a7f-a50d-b46c074e0d79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:19.507157 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa852d8-b172-4a7f-a50d-b46c074e0d79 admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:19.507859 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa852d8-b172-4a7f-a50d-b46c074e0d79 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:49:19.509766 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa852d8-b172-4a7f-a50d-b46c074e0d79 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:49:19.587711 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1183/2355] 10.222.0.81 () {60 vars in 1143 bytes} [Wed Jul 20 22:49:19 2022] POST /identity/v3/users => generated 299 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:49:19.591660 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345e2a3a-9f68-47ea-8d25-a2246079a598 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:49:19.602251 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345e2a3a-9f68-47ea-8d25-a2246079a598 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:49:19.606982 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec3348e-1672-45a1-9d90-bb063dd028c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:49:19.607680 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec3348e-1672-45a1-9d90-bb063dd028c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:19.607970 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec3348e-1672-45a1-9d90-bb063dd028c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:19.608253 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec3348e-1672-45a1-9d90-bb063dd028c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:19.608981 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec3348e-1672-45a1-9d90-bb063dd028c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:49:19.610584 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec3348e-1672-45a1-9d90-bb063dd028c8 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:49:19.621487 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1173/2356] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:49:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:49:19.625316 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa852d8-b172-4a7f-a50d-b46c074e0d79 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:49:19.636727 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa852d8-b172-4a7f-a50d-b46c074e0d79 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:49:19.641442 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfef47e5-20d2-4935-ad85-cf14b66173c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:49:19.642106 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfef47e5-20d2-4935-ad85-cf14b66173c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:19.642395 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfef47e5-20d2-4935-ad85-cf14b66173c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:19.642678 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfef47e5-20d2-4935-ad85-cf14b66173c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:19.643441 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfef47e5-20d2-4935-ad85-cf14b66173c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:49:19.645025 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfef47e5-20d2-4935-ad85-cf14b66173c8 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:49:19.655404 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1184/2357] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:49:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:49:19.659247 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec3348e-1672-45a1-9d90-bb063dd028c8 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:49:19.669274 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec3348e-1672-45a1-9d90-bb063dd028c8 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:49:19.674478 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b86a5c9-e1b1-4b8d-900b-0dede369f55c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:49:19.675537 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b86a5c9-e1b1-4b8d-900b-0dede369f55c admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:19.675821 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b86a5c9-e1b1-4b8d-900b-0dede369f55c admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:19.676098 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b86a5c9-e1b1-4b8d-900b-0dede369f55c admin admin] PATH_INFO: `/v3/projects/468e44bfb9404750a344a13e5df36596/users/184cd5ed00034619a86a9cb48338824f/roles/559942c4de1744169f8b5e0404ecde6b` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:19.691004 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b86a5c9-e1b1-4b8d-900b-0dede369f55c admin admin] RBAC: Authorizing `identity:create_grant(project_id=468e44bfb9404750a344a13e5df36596, user_id=184cd5ed00034619a86a9cb48338824f, role_id=559942c4de1744169f8b5e0404ecde6b)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:49:19.693683 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b86a5c9-e1b1-4b8d-900b-0dede369f55c admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:49:19.770853 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1174/2358] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:49:19 2022] PUT /identity/v3/projects/468e44bfb9404750a344a13e5df36596/users/184cd5ed00034619a86a9cb48338824f/roles/559942c4de1744169f8b5e0404ecde6b => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:49:19.781693 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be3b9d7-a104-44b4-bf4f-324a69abd18e None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:19.784164 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be3b9d7-a104-44b4-bf4f-324a69abd18e None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:19.784164 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be3b9d7-a104-44b4-bf4f-324a69abd18e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:19.809868 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-9be3b9d7-a104-44b4-bf4f-324a69abd18e None None] MFA Rules not processed for user `184cd5ed00034619a86a9cb48338824f`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:49:19.811580 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-9be3b9d7-a104-44b4-bf4f-324a69abd18e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:49:19.824545 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:49:19.876168 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1185/2359] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:49:19 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:49:19.887862 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b86a5c9-e1b1-4b8d-900b-0dede369f55c admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:49:19.889586 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-7b86a5c9-e1b1-4b8d-900b-0dede369f55c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:49:19.921507 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:49:19.952141 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b86a5c9-e1b1-4b8d-900b-0dede369f55c admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:49:19.956120 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0296ff8-832a-494f-b8e4-2e05fb653457 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecf4db289fba4e4a806198c62d31f7c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:49:19.956779 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0296ff8-832a-494f-b8e4-2e05fb653457 service cinder] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:19.957070 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0296ff8-832a-494f-b8e4-2e05fb653457 service cinder] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:19.957355 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0296ff8-832a-494f-b8e4-2e05fb653457 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:19.966072 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0296ff8-832a-494f-b8e4-2e05fb653457 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:49:19.967580 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0296ff8-832a-494f-b8e4-2e05fb653457 service cinder] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:49:19.977312 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1175/2360] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:49:19 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:49:20.002256 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be3b9d7-a104-44b4-bf4f-324a69abd18e None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:49:20.003866 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-9be3b9d7-a104-44b4-bf4f-324a69abd18e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:49:20.077801 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be3b9d7-a104-44b4-bf4f-324a69abd18e None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:49:20.083139 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac57b306-3538-4704-929a-d3dcba8accdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:49:20.084047 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac57b306-3538-4704-929a-d3dcba8accdd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:20.084333 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac57b306-3538-4704-929a-d3dcba8accdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:20.084618 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac57b306-3538-4704-929a-d3dcba8accdd admin admin] PATH_INFO: `/v3/users/fdb4cffdbe9b4b139cceb6d8ee867b8d` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:20.085843 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac57b306-3538-4704-929a-d3dcba8accdd admin admin] RBAC: Authorizing `identity:delete_user(user_id=fdb4cffdbe9b4b139cceb6d8ee867b8d)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:49:20.087821 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac57b306-3538-4704-929a-d3dcba8accdd admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:49:20.350957 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-ac57b306-3538-4704-929a-d3dcba8accdd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fdb4cffdbe9b4b139cceb6d8ee867b8d'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:20.355262 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-ac57b306-3538-4704-929a-d3dcba8accdd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fdb4cffdbe9b4b139cceb6d8ee867b8d'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:20.355262 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-ac57b306-3538-4704-929a-d3dcba8accdd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fdb4cffdbe9b4b139cceb6d8ee867b8d'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:20.359855 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-ac57b306-3538-4704-929a-d3dcba8accdd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fdb4cffdbe9b4b139cceb6d8ee867b8d'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:20.444830 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-ac57b306-3538-4704-929a-d3dcba8accdd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fdb4cffdbe9b4b139cceb6d8ee867b8d'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:20.466457 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1186/2361] 10.222.0.81 () {58 vars in 1223 bytes} [Wed Jul 20 22:49:20 2022] DELETE /identity/v3/users/fdb4cffdbe9b4b139cceb6d8ee867b8d => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:49:20.471899 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a7e27e-2b3e-4993-8c65-f98edc2e1c54 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:20.472524 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a7e27e-2b3e-4993-8c65-f98edc2e1c54 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:20.472524 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a7e27e-2b3e-4993-8c65-f98edc2e1c54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:20.490853 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-36a7e27e-2b3e-4993-8c65-f98edc2e1c54 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:49:20.492127 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-36a7e27e-2b3e-4993-8c65-f98edc2e1c54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:49:20.550934 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1176/2362] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:49:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:49:20.560662 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac57b306-3538-4704-929a-d3dcba8accdd admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:49:20.562037 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-ac57b306-3538-4704-929a-d3dcba8accdd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:49:20.602496 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:49:20.619835 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac57b306-3538-4704-929a-d3dcba8accdd admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:49:20.623483 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5805aa5-af3a-4c33-8555-6bcc1ef3293b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:49:20.624149 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5805aa5-af3a-4c33-8555-6bcc1ef3293b service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:20.624433 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5805aa5-af3a-4c33-8555-6bcc1ef3293b service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:20.624710 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5805aa5-af3a-4c33-8555-6bcc1ef3293b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:20.634882 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5805aa5-af3a-4c33-8555-6bcc1ef3293b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:49:20.636417 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5805aa5-af3a-4c33-8555-6bcc1ef3293b service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:49:20.646943 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1187/2363] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:49:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:49:21.127526 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a7e27e-2b3e-4993-8c65-f98edc2e1c54 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:49:21.129431 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-36a7e27e-2b3e-4993-8c65-f98edc2e1c54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:49:21.145523 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a7e27e-2b3e-4993-8c65-f98edc2e1c54 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:49:21.151490 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3761ea8b-3593-4bd9-b696-980aab8d5692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:49:21.152460 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3761ea8b-3593-4bd9-b696-980aab8d5692 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:21.152819 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3761ea8b-3593-4bd9-b696-980aab8d5692 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:21.153262 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3761ea8b-3593-4bd9-b696-980aab8d5692 admin admin] PATH_INFO: `/v3/projects/fffefe641e7b462aab15453e9611f319` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:21.155451 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3761ea8b-3593-4bd9-b696-980aab8d5692 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fffefe641e7b462aab15453e9611f319)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:49:21.157503 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3761ea8b-3593-4bd9-b696-980aab8d5692 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:49:21.263294 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-3761ea8b-3593-4bd9-b696-980aab8d5692 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fffefe641e7b462aab15453e9611f319', 'request_id': 'req-3761ea8b-3593-4bd9-b696-980aab8d5692'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:21.364376 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-3761ea8b-3593-4bd9-b696-980aab8d5692 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fffefe641e7b462aab15453e9611f319', 'request_id': 'req-3761ea8b-3593-4bd9-b696-980aab8d5692'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:21.370497 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-3761ea8b-3593-4bd9-b696-980aab8d5692 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fffefe641e7b462aab15453e9611f319', 'request_id': 'req-3761ea8b-3593-4bd9-b696-980aab8d5692'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:21.382753 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-3761ea8b-3593-4bd9-b696-980aab8d5692 admin admin] The token cache is being invalidate because project fffefe641e7b462aab15453e9611f319 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:49:21.383137 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-3761ea8b-3593-4bd9-b696-980aab8d5692 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:21.383822 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-3761ea8b-3593-4bd9-b696-980aab8d5692 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:21.385344 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1177/2364] 10.222.0.81 () {58 vars in 1232 bytes} [Wed Jul 20 22:49:21 2022] DELETE /identity/v3/projects/fffefe641e7b462aab15453e9611f319 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:49:23.147192 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5805aa5-af3a-4c33-8555-6bcc1ef3293b service neutron] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:49:23.148591 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-e5805aa5-af3a-4c33-8555-6bcc1ef3293b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:49:23.206809 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5805aa5-af3a-4c33-8555-6bcc1ef3293b service neutron] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:49:23.255532 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0abe6c8-3568-44ca-acab-5d2fb8a641aa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecf4db289fba4e4a806198c62d31f7c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:49:23.256174 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0abe6c8-3568-44ca-acab-5d2fb8a641aa service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:23.256457 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0abe6c8-3568-44ca-acab-5d2fb8a641aa service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:23.256732 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0abe6c8-3568-44ca-acab-5d2fb8a641aa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:23.258500 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-f0abe6c8-3568-44ca-acab-5d2fb8a641aa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:49:23.322532 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0abe6c8-3568-44ca-acab-5d2fb8a641aa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:49:23.324054 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0abe6c8-3568-44ca-acab-5d2fb8a641aa service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:49:23.334696 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1188/2365] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:49:23 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:49:31.123182 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3761ea8b-3593-4bd9-b696-980aab8d5692 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:49:31.124620 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-3761ea8b-3593-4bd9-b696-980aab8d5692 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:49:31.178259 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:49:31.192995 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3761ea8b-3593-4bd9-b696-980aab8d5692 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:49:31.198932 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b09efde-d687-4f31-bacd-25b8b37e26f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:49:31.199896 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b09efde-d687-4f31-bacd-25b8b37e26f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:31.200187 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b09efde-d687-4f31-bacd-25b8b37e26f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:31.200467 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b09efde-d687-4f31-bacd-25b8b37e26f9 admin admin] PATH_INFO: `/v3/users/184cd5ed00034619a86a9cb48338824f` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:31.201671 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b09efde-d687-4f31-bacd-25b8b37e26f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=184cd5ed00034619a86a9cb48338824f)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:49:31.203656 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b09efde-d687-4f31-bacd-25b8b37e26f9 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:49:31.493894 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-3b09efde-d687-4f31-bacd-25b8b37e26f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '184cd5ed00034619a86a9cb48338824f'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:31.494647 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-3b09efde-d687-4f31-bacd-25b8b37e26f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '184cd5ed00034619a86a9cb48338824f'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:31.495688 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-3b09efde-d687-4f31-bacd-25b8b37e26f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '184cd5ed00034619a86a9cb48338824f'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:31.502315 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-3b09efde-d687-4f31-bacd-25b8b37e26f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '184cd5ed00034619a86a9cb48338824f'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:31.576092 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-3b09efde-d687-4f31-bacd-25b8b37e26f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '184cd5ed00034619a86a9cb48338824f'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:31.596295 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1178/2366] 10.222.0.81 () {58 vars in 1223 bytes} [Wed Jul 20 22:49:31 2022] DELETE /identity/v3/users/184cd5ed00034619a86a9cb48338824f => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:49:31.600897 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b40a2df-c4fc-46de-83c5-68e020d06dd6 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:31.601471 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b40a2df-c4fc-46de-83c5-68e020d06dd6 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:31.601471 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b40a2df-c4fc-46de-83c5-68e020d06dd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:31.619050 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-7b40a2df-c4fc-46de-83c5-68e020d06dd6 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:49:31.620499 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-7b40a2df-c4fc-46de-83c5-68e020d06dd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:49:31.663300 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:49:31.682543 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1189/2367] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:49:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 22:49:31.692831 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b09efde-d687-4f31-bacd-25b8b37e26f9 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:49:31.694439 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-3b09efde-d687-4f31-bacd-25b8b37e26f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:49:31.755962 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b09efde-d687-4f31-bacd-25b8b37e26f9 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:49:31.759916 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbce503c-c586-4c84-877e-fb40b3868a07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:49:31.760749 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbce503c-c586-4c84-877e-fb40b3868a07 service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:31.761030 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbce503c-c586-4c84-877e-fb40b3868a07 service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:31.761318 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbce503c-c586-4c84-877e-fb40b3868a07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:31.771029 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbce503c-c586-4c84-877e-fb40b3868a07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:49:31.772521 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbce503c-c586-4c84-877e-fb40b3868a07 service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:49:31.782965 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1179/2368] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:49:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:49:32.207260 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b40a2df-c4fc-46de-83c5-68e020d06dd6 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:49:32.208713 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-7b40a2df-c4fc-46de-83c5-68e020d06dd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:49:32.221688 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b40a2df-c4fc-46de-83c5-68e020d06dd6 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:49:32.226867 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24508b44-e058-4f42-8df7-89a03739e101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:49:32.227847 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24508b44-e058-4f42-8df7-89a03739e101 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:32.228142 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24508b44-e058-4f42-8df7-89a03739e101 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:32.228479 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24508b44-e058-4f42-8df7-89a03739e101 admin admin] PATH_INFO: `/v3/projects/468e44bfb9404750a344a13e5df36596` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:32.238009 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24508b44-e058-4f42-8df7-89a03739e101 admin admin] RBAC: Authorizing `identity:delete_project(project_id=468e44bfb9404750a344a13e5df36596)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:49:32.238009 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24508b44-e058-4f42-8df7-89a03739e101 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:49:32.447188 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-24508b44-e058-4f42-8df7-89a03739e101 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '468e44bfb9404750a344a13e5df36596', 'request_id': 'req-24508b44-e058-4f42-8df7-89a03739e101'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:32.453966 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-24508b44-e058-4f42-8df7-89a03739e101 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '468e44bfb9404750a344a13e5df36596', 'request_id': 'req-24508b44-e058-4f42-8df7-89a03739e101'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:32.609714 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-24508b44-e058-4f42-8df7-89a03739e101 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '468e44bfb9404750a344a13e5df36596', 'request_id': 'req-24508b44-e058-4f42-8df7-89a03739e101'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:32.624372 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-24508b44-e058-4f42-8df7-89a03739e101 admin admin] The token cache is being invalidate because project 468e44bfb9404750a344a13e5df36596 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:49:32.624813 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-24508b44-e058-4f42-8df7-89a03739e101 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:32.625598 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-24508b44-e058-4f42-8df7-89a03739e101 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:32.628415 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1190/2369] 10.222.0.81 () {58 vars in 1232 bytes} [Wed Jul 20 22:49:32 2022] DELETE /identity/v3/projects/468e44bfb9404750a344a13e5df36596 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:49:34.827236 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbce503c-c586-4c84-877e-fb40b3868a07 service neutron] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:49:34.828701 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-fbce503c-c586-4c84-877e-fb40b3868a07 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:49:34.842713 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbce503c-c586-4c84-877e-fb40b3868a07 service neutron] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:49:34.897708 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0145305f-449a-43cc-bd1b-f363c53f944c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:49:34.898605 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0145305f-449a-43cc-bd1b-f363c53f944c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:34.898891 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0145305f-449a-43cc-bd1b-f363c53f944c admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:34.899201 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0145305f-449a-43cc-bd1b-f363c53f944c admin admin] PATH_INFO: `/v3/users/70c89385badc4f5eab13108ca85a2de4` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:34.900719 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0145305f-449a-43cc-bd1b-f363c53f944c admin admin] RBAC: Authorizing `identity:delete_user(user_id=70c89385badc4f5eab13108ca85a2de4)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:49:34.902687 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0145305f-449a-43cc-bd1b-f363c53f944c admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:49:34.916766 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:49:35.104501 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-0145305f-449a-43cc-bd1b-f363c53f944c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70c89385badc4f5eab13108ca85a2de4'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:35.105141 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-0145305f-449a-43cc-bd1b-f363c53f944c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70c89385badc4f5eab13108ca85a2de4'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:35.105742 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-0145305f-449a-43cc-bd1b-f363c53f944c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70c89385badc4f5eab13108ca85a2de4'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:35.113046 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-0145305f-449a-43cc-bd1b-f363c53f944c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70c89385badc4f5eab13108ca85a2de4'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:35.192268 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-0145305f-449a-43cc-bd1b-f363c53f944c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70c89385badc4f5eab13108ca85a2de4'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:35.210955 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1180/2370] 10.222.0.81 () {58 vars in 1223 bytes} [Wed Jul 20 22:49:34 2022] DELETE /identity/v3/users/70c89385badc4f5eab13108ca85a2de4 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:49:35.215554 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc56515-de31-4ab8-bf58-9d2fd0a9143c None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:35.216015 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc56515-de31-4ab8-bf58-9d2fd0a9143c None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:35.216114 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc56515-de31-4ab8-bf58-9d2fd0a9143c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:35.233880 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-6bc56515-de31-4ab8-bf58-9d2fd0a9143c None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:49:35.235106 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-6bc56515-de31-4ab8-bf58-9d2fd0a9143c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:49:35.298451 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1191/2371] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:49:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:49:35.308572 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0145305f-449a-43cc-bd1b-f363c53f944c admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:49:35.310001 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-0145305f-449a-43cc-bd1b-f363c53f944c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:49:35.369843 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0145305f-449a-43cc-bd1b-f363c53f944c admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:49:35.373540 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5853e5df-d3f0-42d7-afc8-9f9dc5ba4ebb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:49:35.374194 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5853e5df-d3f0-42d7-afc8-9f9dc5ba4ebb service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:35.374484 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5853e5df-d3f0-42d7-afc8-9f9dc5ba4ebb service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:35.374768 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5853e5df-d3f0-42d7-afc8-9f9dc5ba4ebb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:35.383776 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5853e5df-d3f0-42d7-afc8-9f9dc5ba4ebb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:49:35.385205 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5853e5df-d3f0-42d7-afc8-9f9dc5ba4ebb service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:49:35.395161 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1181/2372] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:49:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:49:35.783157 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc56515-de31-4ab8-bf58-9d2fd0a9143c None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:49:35.784618 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-6bc56515-de31-4ab8-bf58-9d2fd0a9143c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:49:35.796723 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc56515-de31-4ab8-bf58-9d2fd0a9143c None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:49:35.801997 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60846e01-118e-45b8-8103-8625602b5682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:49:35.802904 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60846e01-118e-45b8-8103-8625602b5682 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:35.803293 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60846e01-118e-45b8-8103-8625602b5682 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:35.803510 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60846e01-118e-45b8-8103-8625602b5682 admin admin] PATH_INFO: `/v3/projects/57fed9e4ecfe40e7a135c45436b207b3` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:35.804811 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60846e01-118e-45b8-8103-8625602b5682 admin admin] RBAC: Authorizing `identity:delete_project(project_id=57fed9e4ecfe40e7a135c45436b207b3)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:49:35.806972 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60846e01-118e-45b8-8103-8625602b5682 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:49:35.901285 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-60846e01-118e-45b8-8103-8625602b5682 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57fed9e4ecfe40e7a135c45436b207b3', 'request_id': 'req-60846e01-118e-45b8-8103-8625602b5682'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:35.907371 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-60846e01-118e-45b8-8103-8625602b5682 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57fed9e4ecfe40e7a135c45436b207b3', 'request_id': 'req-60846e01-118e-45b8-8103-8625602b5682'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:35.969490 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-60846e01-118e-45b8-8103-8625602b5682 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57fed9e4ecfe40e7a135c45436b207b3', 'request_id': 'req-60846e01-118e-45b8-8103-8625602b5682'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:35.972926 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:49:35.981915 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-60846e01-118e-45b8-8103-8625602b5682 admin admin] The token cache is being invalidate because project 57fed9e4ecfe40e7a135c45436b207b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:49:35.982188 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-60846e01-118e-45b8-8103-8625602b5682 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:35.982844 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-60846e01-118e-45b8-8103-8625602b5682 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:35.984553 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1192/2373] 10.222.0.81 () {58 vars in 1232 bytes} [Wed Jul 20 22:49:35 2022] DELETE /identity/v3/projects/57fed9e4ecfe40e7a135c45436b207b3 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:49:35.988627 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5853e5df-d3f0-42d7-afc8-9f9dc5ba4ebb service neutron] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:49:35.998263 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5853e5df-d3f0-42d7-afc8-9f9dc5ba4ebb service neutron] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:49:36.057601 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c956ea-0d6e-4ebc-926d-bd7974c80ba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:49:36.058488 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c956ea-0d6e-4ebc-926d-bd7974c80ba9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:36.058774 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c956ea-0d6e-4ebc-926d-bd7974c80ba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:36.059078 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c956ea-0d6e-4ebc-926d-bd7974c80ba9 admin admin] PATH_INFO: `/v3/users/fee70ddfc8f640dca4875ac178b7344d` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:36.060293 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c956ea-0d6e-4ebc-926d-bd7974c80ba9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fee70ddfc8f640dca4875ac178b7344d)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:49:36.062250 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c956ea-0d6e-4ebc-926d-bd7974c80ba9 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:49:36.221064 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:49:36.229252 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-43c956ea-0d6e-4ebc-926d-bd7974c80ba9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fee70ddfc8f640dca4875ac178b7344d'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:36.229911 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-43c956ea-0d6e-4ebc-926d-bd7974c80ba9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fee70ddfc8f640dca4875ac178b7344d'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:36.230520 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-43c956ea-0d6e-4ebc-926d-bd7974c80ba9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fee70ddfc8f640dca4875ac178b7344d'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:36.237224 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-43c956ea-0d6e-4ebc-926d-bd7974c80ba9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fee70ddfc8f640dca4875ac178b7344d'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:36.329474 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-43c956ea-0d6e-4ebc-926d-bd7974c80ba9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fee70ddfc8f640dca4875ac178b7344d'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:36.348790 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1182/2374] 10.222.0.81 () {58 vars in 1223 bytes} [Wed Jul 20 22:49:35 2022] DELETE /identity/v3/users/fee70ddfc8f640dca4875ac178b7344d => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:49:36.749151 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60846e01-118e-45b8-8103-8625602b5682 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:49:36.751095 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-60846e01-118e-45b8-8103-8625602b5682 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:49:36.765978 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60846e01-118e-45b8-8103-8625602b5682 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:49:36.824414 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b914450e-8a74-4560-8cc7-fa7e1a0ac31b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:49:36.825540 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b914450e-8a74-4560-8cc7-fa7e1a0ac31b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:36.825907 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b914450e-8a74-4560-8cc7-fa7e1a0ac31b admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:36.826250 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b914450e-8a74-4560-8cc7-fa7e1a0ac31b admin admin] PATH_INFO: `/v3/projects/c6bb7979fa9b4fb59f8affa69b9f52c8` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:36.827714 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b914450e-8a74-4560-8cc7-fa7e1a0ac31b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6bb7979fa9b4fb59f8affa69b9f52c8)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:49:36.830547 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b914450e-8a74-4560-8cc7-fa7e1a0ac31b admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:49:36.929311 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-b914450e-8a74-4560-8cc7-fa7e1a0ac31b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6bb7979fa9b4fb59f8affa69b9f52c8', 'request_id': 'req-b914450e-8a74-4560-8cc7-fa7e1a0ac31b'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:36.935449 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-b914450e-8a74-4560-8cc7-fa7e1a0ac31b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6bb7979fa9b4fb59f8affa69b9f52c8', 'request_id': 'req-b914450e-8a74-4560-8cc7-fa7e1a0ac31b'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:37.009405 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-b914450e-8a74-4560-8cc7-fa7e1a0ac31b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6bb7979fa9b4fb59f8affa69b9f52c8', 'request_id': 'req-b914450e-8a74-4560-8cc7-fa7e1a0ac31b'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:37.021309 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-b914450e-8a74-4560-8cc7-fa7e1a0ac31b admin admin] The token cache is being invalidate because project c6bb7979fa9b4fb59f8affa69b9f52c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:49:37.027512 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-b914450e-8a74-4560-8cc7-fa7e1a0ac31b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:37.027512 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-b914450e-8a74-4560-8cc7-fa7e1a0ac31b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:37.027512 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1193/2375] 10.222.0.81 () {58 vars in 1232 bytes} [Wed Jul 20 22:49:36 2022] DELETE /identity/v3/projects/c6bb7979fa9b4fb59f8affa69b9f52c8 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:49:37.031544 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf22c96-f5e6-4133-8cb9-9a9abfd7b7c6 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:37.032001 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf22c96-f5e6-4133-8cb9-9a9abfd7b7c6 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:37.032130 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf22c96-f5e6-4133-8cb9-9a9abfd7b7c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:37.050205 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-ccf22c96-f5e6-4133-8cb9-9a9abfd7b7c6 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:49:37.051459 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-ccf22c96-f5e6-4133-8cb9-9a9abfd7b7c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:49:37.112875 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1183/2376] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:49:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:49:37.122346 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685d16d6-3065-4d91-85a6-705661a2aa7a None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:37.122646 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685d16d6-3065-4d91-85a6-705661a2aa7a None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:37.122932 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685d16d6-3065-4d91-85a6-705661a2aa7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:37.141891 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-685d16d6-3065-4d91-85a6-705661a2aa7a None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:49:37.143224 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-685d16d6-3065-4d91-85a6-705661a2aa7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:49:37.153764 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1194/2377] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:49:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:49:37.158452 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf22c96-f5e6-4133-8cb9-9a9abfd7b7c6 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:49:37.169321 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf22c96-f5e6-4133-8cb9-9a9abfd7b7c6 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:49:37.173896 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cf27e7-5ef0-4c46-9e30-c9ffb0117f7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:49:37.174602 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cf27e7-5ef0-4c46-9e30-c9ffb0117f7c admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:37.174890 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cf27e7-5ef0-4c46-9e30-c9ffb0117f7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:37.175784 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cf27e7-5ef0-4c46-9e30-c9ffb0117f7c admin admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:37.176516 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cf27e7-5ef0-4c46-9e30-c9ffb0117f7c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:49:37.176832 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cf27e7-5ef0-4c46-9e30-c9ffb0117f7c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:49:37.178429 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cf27e7-5ef0-4c46-9e30-c9ffb0117f7c admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:49:37.190109 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1184/2378] 10.222.0.81 () {58 vars in 1152 bytes} [Wed Jul 20 22:49:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:49:37.194390 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685d16d6-3065-4d91-85a6-705661a2aa7a None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:49:37.203629 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685d16d6-3065-4d91-85a6-705661a2aa7a None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:49:37.208260 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541d48f9-09a7-4cd1-bdb3-844bdd800c46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:49:37.208918 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541d48f9-09a7-4cd1-bdb3-844bdd800c46 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:37.209205 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541d48f9-09a7-4cd1-bdb3-844bdd800c46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:37.209483 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541d48f9-09a7-4cd1-bdb3-844bdd800c46 admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:37.210175 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541d48f9-09a7-4cd1-bdb3-844bdd800c46 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:49:37.212225 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541d48f9-09a7-4cd1-bdb3-844bdd800c46 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:49:37.297072 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1195/2379] 10.222.0.81 () {60 vars in 1152 bytes} [Wed Jul 20 22:49:37 2022] POST /identity/v3/projects => generated 373 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 22:49:37.301481 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cf27e7-5ef0-4c46-9e30-c9ffb0117f7c admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:49:37.311555 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cf27e7-5ef0-4c46-9e30-c9ffb0117f7c admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:49:37.370654 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7061d1e-1585-4f7b-8405-9d02ac3543e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:49:37.371370 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7061d1e-1585-4f7b-8405-9d02ac3543e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:37.371658 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7061d1e-1585-4f7b-8405-9d02ac3543e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:37.371939 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7061d1e-1585-4f7b-8405-9d02ac3543e1 admin admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:37.372645 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7061d1e-1585-4f7b-8405-9d02ac3543e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:49:37.374560 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7061d1e-1585-4f7b-8405-9d02ac3543e1 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:49:37.437789 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1185/2380] 10.222.0.81 () {60 vars in 1143 bytes} [Wed Jul 20 22:49:37 2022] POST /identity/v3/users => generated 287 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:49:37.442718 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541d48f9-09a7-4cd1-bdb3-844bdd800c46 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:49:37.453852 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541d48f9-09a7-4cd1-bdb3-844bdd800c46 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:49:37.459783 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4ae27f-4f94-4efe-8a6f-e944228f75e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:49:37.460478 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4ae27f-4f94-4efe-8a6f-e944228f75e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:37.460775 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4ae27f-4f94-4efe-8a6f-e944228f75e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:37.461045 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4ae27f-4f94-4efe-8a6f-e944228f75e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:37.461777 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4ae27f-4f94-4efe-8a6f-e944228f75e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:49:37.463448 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4ae27f-4f94-4efe-8a6f-e944228f75e2 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:49:37.474960 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1196/2381] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:49:37 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:49:37.478938 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7061d1e-1585-4f7b-8405-9d02ac3543e1 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:49:37.489295 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7061d1e-1585-4f7b-8405-9d02ac3543e1 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:49:37.493868 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40a832d-5691-4621-8252-532a92509bcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:49:37.494526 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40a832d-5691-4621-8252-532a92509bcd admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:37.494814 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40a832d-5691-4621-8252-532a92509bcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:37.495125 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40a832d-5691-4621-8252-532a92509bcd admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:37.495850 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40a832d-5691-4621-8252-532a92509bcd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:49:37.497434 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40a832d-5691-4621-8252-532a92509bcd admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:49:37.507820 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1186/2382] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:49:37 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:49:37.511517 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4ae27f-4f94-4efe-8a6f-e944228f75e2 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:49:37.521331 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4ae27f-4f94-4efe-8a6f-e944228f75e2 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:49:37.527132 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30220d93-3701-4c51-849a-2adedb2c10b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:49:37.527916 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30220d93-3701-4c51-849a-2adedb2c10b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:37.528212 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30220d93-3701-4c51-849a-2adedb2c10b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:37.528601 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30220d93-3701-4c51-849a-2adedb2c10b6 admin admin] PATH_INFO: `/v3/projects/ed347fa9b30f4f5eb9e4dd4d29c3f0c4/users/5fe8e726969848f0b7f875defec560b2/roles/559942c4de1744169f8b5e0404ecde6b` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:37.544165 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30220d93-3701-4c51-849a-2adedb2c10b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed347fa9b30f4f5eb9e4dd4d29c3f0c4, user_id=5fe8e726969848f0b7f875defec560b2, role_id=559942c4de1744169f8b5e0404ecde6b)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:49:37.546800 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30220d93-3701-4c51-849a-2adedb2c10b6 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:49:37.604289 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1197/2383] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:49:37 2022] PUT /identity/v3/projects/ed347fa9b30f4f5eb9e4dd4d29c3f0c4/users/5fe8e726969848f0b7f875defec560b2/roles/559942c4de1744169f8b5e0404ecde6b => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:49:37.672125 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6a96c9-26d2-4b7c-9c11-c10d33dd4e8e None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:37.672419 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6a96c9-26d2-4b7c-9c11-c10d33dd4e8e None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:37.672703 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6a96c9-26d2-4b7c-9c11-c10d33dd4e8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:37.699003 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-8b6a96c9-26d2-4b7c-9c11-c10d33dd4e8e None None] MFA Rules not processed for user `5fe8e726969848f0b7f875defec560b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:49:37.700246 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-8b6a96c9-26d2-4b7c-9c11-c10d33dd4e8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:49:37.708405 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:49:37.766496 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1187/2384] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:49:37 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:49:37.778252 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30220d93-3701-4c51-849a-2adedb2c10b6 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:49:37.780059 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-30220d93-3701-4c51-849a-2adedb2c10b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:49:37.803472 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=89,keystone:DELETE=8,keystone:INSERT=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:49:37.846864 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30220d93-3701-4c51-849a-2adedb2c10b6 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:49:37.851188 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3511fe4-b7ec-4a00-bcb0-fe5110d8296a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecf4db289fba4e4a806198c62d31f7c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:49:37.851965 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3511fe4-b7ec-4a00-bcb0-fe5110d8296a service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:37.852330 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3511fe4-b7ec-4a00-bcb0-fe5110d8296a service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:37.852671 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3511fe4-b7ec-4a00-bcb0-fe5110d8296a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:37.864488 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3511fe4-b7ec-4a00-bcb0-fe5110d8296a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:49:37.890609 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3511fe4-b7ec-4a00-bcb0-fe5110d8296a service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:49:37.890609 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1198/2385] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:49:37 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:49:48.009581 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=23 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:49:48.300619 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=18 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:49:52.286388 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6a96c9-26d2-4b7c-9c11-c10d33dd4e8e None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:49:52.350956 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6a96c9-26d2-4b7c-9c11-c10d33dd4e8e None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:49:52.355546 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655dc363-ffcc-4593-81cb-9146ca82d0f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:49:52.356438 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655dc363-ffcc-4593-81cb-9146ca82d0f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:52.356724 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655dc363-ffcc-4593-81cb-9146ca82d0f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:52.357010 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655dc363-ffcc-4593-81cb-9146ca82d0f1 admin admin] PATH_INFO: `/v3/users/5fe8e726969848f0b7f875defec560b2` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:52.358228 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655dc363-ffcc-4593-81cb-9146ca82d0f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5fe8e726969848f0b7f875defec560b2)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:49:52.360183 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655dc363-ffcc-4593-81cb-9146ca82d0f1 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:49:52.537710 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-655dc363-ffcc-4593-81cb-9146ca82d0f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5fe8e726969848f0b7f875defec560b2'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:52.538393 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-655dc363-ffcc-4593-81cb-9146ca82d0f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5fe8e726969848f0b7f875defec560b2'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:52.539040 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-655dc363-ffcc-4593-81cb-9146ca82d0f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5fe8e726969848f0b7f875defec560b2'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:52.546789 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-655dc363-ffcc-4593-81cb-9146ca82d0f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5fe8e726969848f0b7f875defec560b2'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:52.625779 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-655dc363-ffcc-4593-81cb-9146ca82d0f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5fe8e726969848f0b7f875defec560b2'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:52.644567 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1188/2386] 10.222.0.81 () {58 vars in 1223 bytes} [Wed Jul 20 22:49:52 2022] DELETE /identity/v3/users/5fe8e726969848f0b7f875defec560b2 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:49:52.649265 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff5e32b-9400-4800-9d5b-75cce0ccf411 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:52.649959 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff5e32b-9400-4800-9d5b-75cce0ccf411 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:52.650315 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff5e32b-9400-4800-9d5b-75cce0ccf411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:52.671224 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-0ff5e32b-9400-4800-9d5b-75cce0ccf411 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:49:52.672456 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-0ff5e32b-9400-4800-9d5b-75cce0ccf411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:49:52.739883 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1199/2387] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:49:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:49:52.749464 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655dc363-ffcc-4593-81cb-9146ca82d0f1 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:49:52.750873 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-655dc363-ffcc-4593-81cb-9146ca82d0f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:49:52.811589 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655dc363-ffcc-4593-81cb-9146ca82d0f1 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:49:52.815836 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fd61ad-89b1-4fd9-948b-4bab00846310 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:49:52.816494 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fd61ad-89b1-4fd9-948b-4bab00846310 service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:52.816780 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fd61ad-89b1-4fd9-948b-4bab00846310 service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:52.817053 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fd61ad-89b1-4fd9-948b-4bab00846310 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:52.826125 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fd61ad-89b1-4fd9-948b-4bab00846310 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:49:52.827594 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fd61ad-89b1-4fd9-948b-4bab00846310 service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:49:52.837647 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1189/2388] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:49:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:49:53.238563 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff5e32b-9400-4800-9d5b-75cce0ccf411 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:49:53.239993 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-0ff5e32b-9400-4800-9d5b-75cce0ccf411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:49:53.253382 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff5e32b-9400-4800-9d5b-75cce0ccf411 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:49:53.258095 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa812dc1-1049-4d46-b5a1-05a7da40417f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:49:53.259093 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa812dc1-1049-4d46-b5a1-05a7da40417f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:49:53.259353 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa812dc1-1049-4d46-b5a1-05a7da40417f admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:49:53.259636 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa812dc1-1049-4d46-b5a1-05a7da40417f admin admin] PATH_INFO: `/v3/projects/ed347fa9b30f4f5eb9e4dd4d29c3f0c4` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:49:53.260848 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa812dc1-1049-4d46-b5a1-05a7da40417f admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed347fa9b30f4f5eb9e4dd4d29c3f0c4)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:49:53.262763 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa812dc1-1049-4d46-b5a1-05a7da40417f admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:49:53.386453 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-fa812dc1-1049-4d46-b5a1-05a7da40417f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed347fa9b30f4f5eb9e4dd4d29c3f0c4', 'request_id': 'req-fa812dc1-1049-4d46-b5a1-05a7da40417f'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:53.392670 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-fa812dc1-1049-4d46-b5a1-05a7da40417f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed347fa9b30f4f5eb9e4dd4d29c3f0c4', 'request_id': 'req-fa812dc1-1049-4d46-b5a1-05a7da40417f'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:53.448447 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-fa812dc1-1049-4d46-b5a1-05a7da40417f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed347fa9b30f4f5eb9e4dd4d29c3f0c4', 'request_id': 'req-fa812dc1-1049-4d46-b5a1-05a7da40417f'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:53.460012 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-fa812dc1-1049-4d46-b5a1-05a7da40417f admin admin] The token cache is being invalidate because project ed347fa9b30f4f5eb9e4dd4d29c3f0c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:49:53.460305 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-fa812dc1-1049-4d46-b5a1-05a7da40417f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:53.460935 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-fa812dc1-1049-4d46-b5a1-05a7da40417f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:49:53.462486 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1200/2389] 10.222.0.81 () {58 vars in 1232 bytes} [Wed Jul 20 22:49:53 2022] DELETE /identity/v3/projects/ed347fa9b30f4f5eb9e4dd4d29c3f0c4 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:50:02.422509 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82896c6-c71f-4337-b602-1a540c70909c None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:50:02.423003 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82896c6-c71f-4337-b602-1a540c70909c None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:50:02.423147 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82896c6-c71f-4337-b602-1a540c70909c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:50:02.441223 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-b82896c6-c71f-4337-b602-1a540c70909c None None] MFA Rules not processed for user `3e072e59bf3e46aaa062ec85b0e2c91a`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:50:02.442435 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-b82896c6-c71f-4337-b602-1a540c70909c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:50:02.459251 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:50:02.496911 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1190/2390] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:50:02 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:50:03.456753 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:50:12.556572 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=15 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:50:24.222401 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa812dc1-1049-4d46-b5a1-05a7da40417f admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:50:24.223925 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-fa812dc1-1049-4d46-b5a1-05a7da40417f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:50:24.279905 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa812dc1-1049-4d46-b5a1-05a7da40417f admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:50:24.283503 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737975f3-07e4-458e-9a3c-ef19c30c82a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:50:24.284152 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737975f3-07e4-458e-9a3c-ef19c30c82a4 service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:50:24.284442 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737975f3-07e4-458e-9a3c-ef19c30c82a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:50:24.284724 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737975f3-07e4-458e-9a3c-ef19c30c82a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:50:24.286505 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-737975f3-07e4-458e-9a3c-ef19c30c82a4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:50:24.355492 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737975f3-07e4-458e-9a3c-ef19c30c82a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:50:24.356931 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737975f3-07e4-458e-9a3c-ef19c30c82a4 service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:50:24.372064 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1201/2391] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:50:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:50:25.544325 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82896c6-c71f-4337-b602-1a540c70909c None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:50:25.545778 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-b82896c6-c71f-4337-b602-1a540c70909c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:50:25.557534 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82896c6-c71f-4337-b602-1a540c70909c None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:50:25.561091 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f984629e-444a-4813-b518-58acf88ce235 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:50:25.561762 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f984629e-444a-4813-b518-58acf88ce235 service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:50:25.562052 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f984629e-444a-4813-b518-58acf88ce235 service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:50:25.562334 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f984629e-444a-4813-b518-58acf88ce235 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:50:25.564062 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-f984629e-444a-4813-b518-58acf88ce235 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:50:25.577527 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f984629e-444a-4813-b518-58acf88ce235 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:50:25.578953 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f984629e-444a-4813-b518-58acf88ce235 service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:50:25.590384 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1191/2392] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:50:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:50:34.364751 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=54 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:50:35.584916 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=6 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:50:47.638139 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737975f3-07e4-458e-9a3c-ef19c30c82a4 service neutron] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:50:47.639679 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-737975f3-07e4-458e-9a3c-ef19c30c82a4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:50:47.654196 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737975f3-07e4-458e-9a3c-ef19c30c82a4 service neutron] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:50:47.659852 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cfdd32-eaa8-4bfd-b7cf-9cde531c85a9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7829324db2574d38b18c749d0559177f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:50:47.660497 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cfdd32-eaa8-4bfd-b7cf-9cde531c85a9 service nova] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:50:47.660789 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cfdd32-eaa8-4bfd-b7cf-9cde531c85a9 service nova] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:50:47.661070 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cfdd32-eaa8-4bfd-b7cf-9cde531c85a9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:50:47.662825 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-f7cfdd32-eaa8-4bfd-b7cf-9cde531c85a9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:50:47.720515 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cfdd32-eaa8-4bfd-b7cf-9cde531c85a9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:50:47.722099 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cfdd32-eaa8-4bfd-b7cf-9cde531c85a9 service nova] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:50:47.732395 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1202/2393] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:50:47 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:50:52.477244 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64da7e93-45b6-46a9-85b5-c5ac3a4c85c8 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:50:52.477863 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64da7e93-45b6-46a9-85b5-c5ac3a4c85c8 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:50:52.477863 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64da7e93-45b6-46a9-85b5-c5ac3a4c85c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:50:52.495104 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-64da7e93-45b6-46a9-85b5-c5ac3a4c85c8 None None] MFA Rules not processed for user `1f237b9adae24d1ca2262d89eb000de4`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:50:52.496385 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-64da7e93-45b6-46a9-85b5-c5ac3a4c85c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:50:52.551391 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1192/2394] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:50:52 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:50:52.794985 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cfdd32-eaa8-4bfd-b7cf-9cde531c85a9 service nova] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:50:52.804543 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cfdd32-eaa8-4bfd-b7cf-9cde531c85a9 service nova] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:50:52.808712 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40be14e-c28c-4fda-a78c-5721385d4353 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:50:52.809688 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40be14e-c28c-4fda-a78c-5721385d4353 service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:50:52.809982 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40be14e-c28c-4fda-a78c-5721385d4353 service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:50:52.810273 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40be14e-c28c-4fda-a78c-5721385d4353 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:50:52.812080 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-e40be14e-c28c-4fda-a78c-5721385d4353 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:50:52.822208 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40be14e-c28c-4fda-a78c-5721385d4353 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:50:52.823704 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40be14e-c28c-4fda-a78c-5721385d4353 service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:50:52.832083 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1203/2395] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:50:52 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:50:53.220762 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64da7e93-45b6-46a9-85b5-c5ac3a4c85c8 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:50:53.222254 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-64da7e93-45b6-46a9-85b5-c5ac3a4c85c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:50:53.296312 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64da7e93-45b6-46a9-85b5-c5ac3a4c85c8 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:50:53.306922 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6fa786-090d-4f0a-babb-3e069897d59a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5500cebdd9d4f8e9aecb45d6c955878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:50:53.307596 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6fa786-090d-4f0a-babb-3e069897d59a service placement] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:50:53.307880 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6fa786-090d-4f0a-babb-3e069897d59a service placement] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:50:53.308178 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6fa786-090d-4f0a-babb-3e069897d59a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:50:53.309919 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-8f6fa786-090d-4f0a-babb-3e069897d59a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:50:53.322010 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6fa786-090d-4f0a-babb-3e069897d59a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:50:53.323619 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6fa786-090d-4f0a-babb-3e069897d59a service placement] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:50:53.333311 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1193/2396] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:50:53 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:51:02.828492 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=34 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:51:03.329178 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=57 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:51:07.365426 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40be14e-c28c-4fda-a78c-5721385d4353 service neutron] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:51:07.374528 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40be14e-c28c-4fda-a78c-5721385d4353 service neutron] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:51:07.378185 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef653f3-c633-4502-8de5-896f009eacb9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:51:07.378831 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef653f3-c633-4502-8de5-896f009eacb9 service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:51:07.379156 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef653f3-c633-4502-8de5-896f009eacb9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:51:07.379440 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef653f3-c633-4502-8de5-896f009eacb9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:51:07.381210 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-6ef653f3-c633-4502-8de5-896f009eacb9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:51:07.391444 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef653f3-c633-4502-8de5-896f009eacb9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:51:07.392874 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef653f3-c633-4502-8de5-896f009eacb9 service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:51:07.401141 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1204/2397] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:51:07 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:51:17.397643 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=6 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:51:22.230799 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6fa786-090d-4f0a-babb-3e069897d59a service placement] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:51:22.239233 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-8f6fa786-090d-4f0a-babb-3e069897d59a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:51:22.248306 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6fa786-090d-4f0a-babb-3e069897d59a service placement] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:51:22.253807 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ae0b77-20e0-4d8d-a9f7-581a0597c5d7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7829324db2574d38b18c749d0559177f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:51:22.254446 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ae0b77-20e0-4d8d-a9f7-581a0597c5d7 service nova] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:51:22.254734 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ae0b77-20e0-4d8d-a9f7-581a0597c5d7 service nova] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:51:22.255038 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ae0b77-20e0-4d8d-a9f7-581a0597c5d7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:51:22.256764 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-85ae0b77-20e0-4d8d-a9f7-581a0597c5d7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:51:22.316177 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ae0b77-20e0-4d8d-a9f7-581a0597c5d7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:51:22.317869 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ae0b77-20e0-4d8d-a9f7-581a0597c5d7 service nova] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:51:22.327860 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1194/2398] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:51:22 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:51:28.712735 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a52329-28c4-4e93-b81d-1cbe37126941 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:51:28.713240 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a52329-28c4-4e93-b81d-1cbe37126941 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:51:28.713343 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a52329-28c4-4e93-b81d-1cbe37126941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:51:28.739324 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-c2a52329-28c4-4e93-b81d-1cbe37126941 None None] MFA Rules not processed for user `1ed9249b2a9b42d9972ed71c04f2c888`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:51:28.739324 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-c2a52329-28c4-4e93-b81d-1cbe37126941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:51:28.739324 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1205/2399] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:51:28 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:51:32.322856 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=28 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:51:38.725313 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:51:39.899367 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ae0b77-20e0-4d8d-a9f7-581a0597c5d7 service nova] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:51:39.921674 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ae0b77-20e0-4d8d-a9f7-581a0597c5d7 service nova] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:51:39.933118 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9f37b6-09c7-4665-94cc-7ec18495334c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7829324db2574d38b18c749d0559177f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:51:39.934010 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9f37b6-09c7-4665-94cc-7ec18495334c service nova] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:51:39.934484 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9f37b6-09c7-4665-94cc-7ec18495334c service nova] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:51:39.935061 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9f37b6-09c7-4665-94cc-7ec18495334c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:51:39.937779 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-6c9f37b6-09c7-4665-94cc-7ec18495334c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:51:39.951637 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9f37b6-09c7-4665-94cc-7ec18495334c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:51:39.953925 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9f37b6-09c7-4665-94cc-7ec18495334c service nova] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:51:39.968473 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1195/2400] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:51:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:51:45.206051 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a52329-28c4-4e93-b81d-1cbe37126941 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:51:45.218783 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a52329-28c4-4e93-b81d-1cbe37126941 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:51:45.225961 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53548606-338d-454e-a957-409da697b138 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7829324db2574d38b18c749d0559177f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:51:45.228286 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53548606-338d-454e-a957-409da697b138 service nova] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:51:45.228286 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53548606-338d-454e-a957-409da697b138 service nova] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:51:45.228286 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53548606-338d-454e-a957-409da697b138 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:51:45.229074 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-53548606-338d-454e-a957-409da697b138 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:51:45.241395 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53548606-338d-454e-a957-409da697b138 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:51:45.243023 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53548606-338d-454e-a957-409da697b138 service nova] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:51:45.253989 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1206/2401] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:51:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:51:49.962284 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=6 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:51:55.249558 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=6 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:52:17.953660 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9f37b6-09c7-4665-94cc-7ec18495334c service nova] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:52:17.965494 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9f37b6-09c7-4665-94cc-7ec18495334c service nova] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:52:17.971116 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00073534-ea82-4ea3-b659-b706aa395e1a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5500cebdd9d4f8e9aecb45d6c955878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:52:17.971768 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00073534-ea82-4ea3-b659-b706aa395e1a service placement] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:52:17.972050 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00073534-ea82-4ea3-b659-b706aa395e1a service placement] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:52:17.972327 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00073534-ea82-4ea3-b659-b706aa395e1a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:52:17.974097 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-00073534-ea82-4ea3-b659-b706aa395e1a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:52:17.985700 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00073534-ea82-4ea3-b659-b706aa395e1a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:52:17.987299 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00073534-ea82-4ea3-b659-b706aa395e1a service placement] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:52:17.997209 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1196/2402] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:52:17 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:52:18.088917 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53548606-338d-454e-a957-409da697b138 service nova] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:52:18.091029 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-53548606-338d-454e-a957-409da697b138 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:52:18.155451 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53548606-338d-454e-a957-409da697b138 service nova] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:52:18.160071 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b1fac0-69e8-4eca-89c3-626fb92e931b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:52:18.160959 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b1fac0-69e8-4eca-89c3-626fb92e931b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:52:18.161272 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b1fac0-69e8-4eca-89c3-626fb92e931b admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:52:18.161551 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b1fac0-69e8-4eca-89c3-626fb92e931b admin admin] PATH_INFO: `/v3/users/1f237b9adae24d1ca2262d89eb000de4` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:52:18.162745 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b1fac0-69e8-4eca-89c3-626fb92e931b admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f237b9adae24d1ca2262d89eb000de4)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:52:18.164709 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b1fac0-69e8-4eca-89c3-626fb92e931b admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:52:18.356281 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-c9b1fac0-69e8-4eca-89c3-626fb92e931b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f237b9adae24d1ca2262d89eb000de4'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:52:18.356942 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-c9b1fac0-69e8-4eca-89c3-626fb92e931b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f237b9adae24d1ca2262d89eb000de4'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:52:18.357547 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-c9b1fac0-69e8-4eca-89c3-626fb92e931b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f237b9adae24d1ca2262d89eb000de4'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:52:18.364420 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-c9b1fac0-69e8-4eca-89c3-626fb92e931b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f237b9adae24d1ca2262d89eb000de4'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:52:18.423300 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-c9b1fac0-69e8-4eca-89c3-626fb92e931b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f237b9adae24d1ca2262d89eb000de4'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:52:18.441112 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1207/2403] 10.222.0.81 () {58 vars in 1223 bytes} [Wed Jul 20 22:52:18 2022] DELETE /identity/v3/users/1f237b9adae24d1ca2262d89eb000de4 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:52:18.682004 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00073534-ea82-4ea3-b659-b706aa395e1a service placement] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:52:18.693596 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00073534-ea82-4ea3-b659-b706aa395e1a service placement] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:52:18.753233 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e5e5fe-8214-4588-8731-58f5f022163c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:52:18.754113 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e5e5fe-8214-4588-8731-58f5f022163c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:52:18.754401 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e5e5fe-8214-4588-8731-58f5f022163c admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:52:18.754703 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e5e5fe-8214-4588-8731-58f5f022163c admin admin] PATH_INFO: `/v3/projects/650d54a3145d42748ed81457cdd1eb34` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:52:18.755893 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e5e5fe-8214-4588-8731-58f5f022163c admin admin] RBAC: Authorizing `identity:delete_project(project_id=650d54a3145d42748ed81457cdd1eb34)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:52:18.757833 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e5e5fe-8214-4588-8731-58f5f022163c admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:52:18.858660 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-61e5e5fe-8214-4588-8731-58f5f022163c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '650d54a3145d42748ed81457cdd1eb34', 'request_id': 'req-61e5e5fe-8214-4588-8731-58f5f022163c'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:52:18.955956 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-61e5e5fe-8214-4588-8731-58f5f022163c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '650d54a3145d42748ed81457cdd1eb34', 'request_id': 'req-61e5e5fe-8214-4588-8731-58f5f022163c'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:52:18.963083 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-61e5e5fe-8214-4588-8731-58f5f022163c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '650d54a3145d42748ed81457cdd1eb34', 'request_id': 'req-61e5e5fe-8214-4588-8731-58f5f022163c'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:52:18.975077 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-61e5e5fe-8214-4588-8731-58f5f022163c admin admin] The token cache is being invalidate because project 650d54a3145d42748ed81457cdd1eb34 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:52:18.975374 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-61e5e5fe-8214-4588-8731-58f5f022163c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:52:18.976030 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-61e5e5fe-8214-4588-8731-58f5f022163c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:52:18.977570 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1197/2404] 10.222.0.81 () {58 vars in 1232 bytes} [Wed Jul 20 22:52:18 2022] DELETE /identity/v3/projects/650d54a3145d42748ed81457cdd1eb34 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:52:18.982987 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efc9591-2ce8-47ec-a434-ec0067979664 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:52:18.983932 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efc9591-2ce8-47ec-a434-ec0067979664 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:52:18.990662 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efc9591-2ce8-47ec-a434-ec0067979664 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:52:19.002260 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-1efc9591-2ce8-47ec-a434-ec0067979664 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:52:19.003507 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-1efc9591-2ce8-47ec-a434-ec0067979664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:52:19.062592 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1208/2405] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:52:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:52:19.070484 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f836c98-a0dc-4723-a544-390e3e958161 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:52:19.070798 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f836c98-a0dc-4723-a544-390e3e958161 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:52:19.071132 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f836c98-a0dc-4723-a544-390e3e958161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:52:19.088935 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-2f836c98-a0dc-4723-a544-390e3e958161 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:52:19.090136 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-2f836c98-a0dc-4723-a544-390e3e958161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:52:19.098569 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1198/2406] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:52:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:52:19.102695 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efc9591-2ce8-47ec-a434-ec0067979664 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:52:19.113456 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efc9591-2ce8-47ec-a434-ec0067979664 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:52:19.118379 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33648f0d-c0dc-47d5-8afa-2065ae7cafcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:52:19.119084 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33648f0d-c0dc-47d5-8afa-2065ae7cafcc admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:52:19.119375 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33648f0d-c0dc-47d5-8afa-2065ae7cafcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:52:19.119657 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33648f0d-c0dc-47d5-8afa-2065ae7cafcc admin admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:52:19.120408 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33648f0d-c0dc-47d5-8afa-2065ae7cafcc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:52:19.120723 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33648f0d-c0dc-47d5-8afa-2065ae7cafcc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:52:19.122327 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33648f0d-c0dc-47d5-8afa-2065ae7cafcc admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:52:19.134265 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1209/2407] 10.222.0.81 () {58 vars in 1152 bytes} [Wed Jul 20 22:52:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:52:19.138618 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f836c98-a0dc-4723-a544-390e3e958161 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:52:19.148156 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f836c98-a0dc-4723-a544-390e3e958161 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:52:19.153767 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13884ad9-17d1-46ef-a4fd-2b6a4a85adfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:52:19.154424 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13884ad9-17d1-46ef-a4fd-2b6a4a85adfb admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:52:19.154707 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13884ad9-17d1-46ef-a4fd-2b6a4a85adfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:52:19.154984 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13884ad9-17d1-46ef-a4fd-2b6a4a85adfb admin admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:52:19.155697 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13884ad9-17d1-46ef-a4fd-2b6a4a85adfb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:52:19.157606 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13884ad9-17d1-46ef-a4fd-2b6a4a85adfb admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:52:19.242622 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1199/2408] 10.222.0.81 () {60 vars in 1152 bytes} [Wed Jul 20 22:52:19 2022] POST /identity/v3/projects => generated 373 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:52:19.246725 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33648f0d-c0dc-47d5-8afa-2065ae7cafcc admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:52:19.256589 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33648f0d-c0dc-47d5-8afa-2065ae7cafcc admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:52:19.260086 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:52:19.313839 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1750a94-b0c4-413b-bd2a-4cc561eedbde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:52:19.316150 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1750a94-b0c4-413b-bd2a-4cc561eedbde admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:52:19.316781 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1750a94-b0c4-413b-bd2a-4cc561eedbde admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:52:19.317421 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1750a94-b0c4-413b-bd2a-4cc561eedbde admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:52:19.319249 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1750a94-b0c4-413b-bd2a-4cc561eedbde admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:52:19.321952 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1750a94-b0c4-413b-bd2a-4cc561eedbde admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:52:19.406641 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1210/2409] 10.222.0.81 () {60 vars in 1143 bytes} [Wed Jul 20 22:52:19 2022] POST /identity/v3/users => generated 287 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:52:19.410566 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13884ad9-17d1-46ef-a4fd-2b6a4a85adfb admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:52:19.420855 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13884ad9-17d1-46ef-a4fd-2b6a4a85adfb admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:52:19.425515 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb502e62-170d-4652-b88e-43299aa0bf71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:52:19.426181 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb502e62-170d-4652-b88e-43299aa0bf71 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:52:19.426469 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb502e62-170d-4652-b88e-43299aa0bf71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:52:19.426755 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb502e62-170d-4652-b88e-43299aa0bf71 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:52:19.427518 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb502e62-170d-4652-b88e-43299aa0bf71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:52:19.429116 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb502e62-170d-4652-b88e-43299aa0bf71 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:52:19.441180 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1200/2410] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:52:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:52:19.445099 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1750a94-b0c4-413b-bd2a-4cc561eedbde admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:52:19.455577 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1750a94-b0c4-413b-bd2a-4cc561eedbde admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:52:19.460165 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d01cf68-dcb7-44ab-a98d-af84c9542023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:52:19.460822 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d01cf68-dcb7-44ab-a98d-af84c9542023 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:52:19.461108 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d01cf68-dcb7-44ab-a98d-af84c9542023 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:52:19.461412 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d01cf68-dcb7-44ab-a98d-af84c9542023 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:52:19.462126 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d01cf68-dcb7-44ab-a98d-af84c9542023 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:52:19.463753 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d01cf68-dcb7-44ab-a98d-af84c9542023 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:52:19.474071 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1211/2411] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:52:19 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:52:19.478220 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb502e62-170d-4652-b88e-43299aa0bf71 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:52:19.488125 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb502e62-170d-4652-b88e-43299aa0bf71 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:52:19.492716 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dace4b-a9ac-43dc-827a-0b741d250856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:52:19.493533 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dace4b-a9ac-43dc-827a-0b741d250856 admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:52:19.493814 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dace4b-a9ac-43dc-827a-0b741d250856 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:52:19.494089 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dace4b-a9ac-43dc-827a-0b741d250856 admin admin] PATH_INFO: `/v3/projects/a257ecd598a74a5cb15fc4b62c5a5979/users/3ce4c02d3e314761ba613a802194b709/roles/559942c4de1744169f8b5e0404ecde6b` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:52:19.632753 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dace4b-a9ac-43dc-827a-0b741d250856 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a257ecd598a74a5cb15fc4b62c5a5979, user_id=3ce4c02d3e314761ba613a802194b709, role_id=559942c4de1744169f8b5e0404ecde6b)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:52:19.635526 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dace4b-a9ac-43dc-827a-0b741d250856 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:52:19.703809 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1201/2412] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:52:19 2022] PUT /identity/v3/projects/a257ecd598a74a5cb15fc4b62c5a5979/users/3ce4c02d3e314761ba613a802194b709/roles/559942c4de1744169f8b5e0404ecde6b => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:52:19.708903 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81770769-7867-4142-996e-9534cc8ef527 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:52:19.709204 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81770769-7867-4142-996e-9534cc8ef527 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:52:19.709489 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81770769-7867-4142-996e-9534cc8ef527 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:52:19.727072 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-81770769-7867-4142-996e-9534cc8ef527 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:52:19.728332 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-81770769-7867-4142-996e-9534cc8ef527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:52:19.786877 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1212/2413] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:52:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:52:19.796397 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dace4b-a9ac-43dc-827a-0b741d250856 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:52:19.798169 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-27dace4b-a9ac-43dc-827a-0b741d250856 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:52:19.813255 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:52:19.854984 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dace4b-a9ac-43dc-827a-0b741d250856 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:52:19.858825 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd255cb-336d-4ded-b3bb-23cdfd1aef02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:52:19.859490 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd255cb-336d-4ded-b3bb-23cdfd1aef02 service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:52:19.859916 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd255cb-336d-4ded-b3bb-23cdfd1aef02 service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:52:19.860197 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd255cb-336d-4ded-b3bb-23cdfd1aef02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:52:19.869051 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd255cb-336d-4ded-b3bb-23cdfd1aef02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:52:19.870490 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd255cb-336d-4ded-b3bb-23cdfd1aef02 service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:52:19.880635 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1202/2414] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:52:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:52:24.906868 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6f3cd8-b9c7-4a79-b3e1-5db929f46d72 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:52:24.907460 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6f3cd8-b9c7-4a79-b3e1-5db929f46d72 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:52:24.907460 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6f3cd8-b9c7-4a79-b3e1-5db929f46d72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:52:24.939581 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-1d6f3cd8-b9c7-4a79-b3e1-5db929f46d72 None None] MFA Rules not processed for user `3ce4c02d3e314761ba613a802194b709`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:52:24.940456 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-1d6f3cd8-b9c7-4a79-b3e1-5db929f46d72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:52:25.012252 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1213/2415] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:52:24 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:52:25.016887 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd255cb-336d-4ded-b3bb-23cdfd1aef02 service neutron] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:52:25.027259 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd255cb-336d-4ded-b3bb-23cdfd1aef02 service neutron] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:52:25.031938 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab289350-92cf-47dd-a987-dc47f1679bde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:52:25.032748 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab289350-92cf-47dd-a987-dc47f1679bde admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:52:25.033036 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab289350-92cf-47dd-a987-dc47f1679bde admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:52:25.033315 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab289350-92cf-47dd-a987-dc47f1679bde admin admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:52:25.034018 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab289350-92cf-47dd-a987-dc47f1679bde admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:52:25.035956 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab289350-92cf-47dd-a987-dc47f1679bde admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:52:25.139865 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1203/2416] 10.222.0.81 () {60 vars in 1152 bytes} [Wed Jul 20 22:52:25 2022] POST /identity/v3/projects => generated 373 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 22:52:25.144523 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6f3cd8-b9c7-4a79-b3e1-5db929f46d72 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:52:25.155694 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6f3cd8-b9c7-4a79-b3e1-5db929f46d72 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:52:25.169060 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:52:25.215530 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1e9ed6-8cb9-41d7-a0ae-2d876f4ff85b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:52:25.216215 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1e9ed6-8cb9-41d7-a0ae-2d876f4ff85b admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:52:25.216505 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1e9ed6-8cb9-41d7-a0ae-2d876f4ff85b admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:52:25.216784 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1e9ed6-8cb9-41d7-a0ae-2d876f4ff85b admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:52:25.217505 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1e9ed6-8cb9-41d7-a0ae-2d876f4ff85b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:52:25.219459 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1e9ed6-8cb9-41d7-a0ae-2d876f4ff85b admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:52:25.321226 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1214/2417] 10.222.0.81 () {60 vars in 1143 bytes} [Wed Jul 20 22:52:25 2022] POST /identity/v3/users => generated 293 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:52:25.325334 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab289350-92cf-47dd-a987-dc47f1679bde admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:52:25.335720 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab289350-92cf-47dd-a987-dc47f1679bde admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:52:25.340809 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c115b2-6753-4247-8a4b-ebd3b90fb46f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:52:25.341468 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c115b2-6753-4247-8a4b-ebd3b90fb46f admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:52:25.341955 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c115b2-6753-4247-8a4b-ebd3b90fb46f admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:52:25.342069 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c115b2-6753-4247-8a4b-ebd3b90fb46f admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:52:25.342792 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c115b2-6753-4247-8a4b-ebd3b90fb46f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:52:25.344429 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c115b2-6753-4247-8a4b-ebd3b90fb46f admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:52:25.354749 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1204/2418] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:52:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:52:25.359398 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1e9ed6-8cb9-41d7-a0ae-2d876f4ff85b admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:52:25.374173 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1e9ed6-8cb9-41d7-a0ae-2d876f4ff85b admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:52:25.378818 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4896d636-6861-4a29-bb99-27efec1d6bf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:52:25.379766 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4896d636-6861-4a29-bb99-27efec1d6bf3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:52:25.380054 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4896d636-6861-4a29-bb99-27efec1d6bf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:52:25.380334 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4896d636-6861-4a29-bb99-27efec1d6bf3 admin admin] PATH_INFO: `/v3/projects/0e9874be0cd542829242caded150fd52/users/020bbaed1f9148c4a9daa75180935b8e/roles/d89b6c92d35a401a8517f1d47f6be843` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:52:25.397611 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4896d636-6861-4a29-bb99-27efec1d6bf3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e9874be0cd542829242caded150fd52, user_id=020bbaed1f9148c4a9daa75180935b8e, role_id=d89b6c92d35a401a8517f1d47f6be843)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:52:25.400293 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4896d636-6861-4a29-bb99-27efec1d6bf3 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:52:25.521119 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1215/2419] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:52:25 2022] PUT /identity/v3/projects/0e9874be0cd542829242caded150fd52/users/020bbaed1f9148c4a9daa75180935b8e/roles/d89b6c92d35a401a8517f1d47f6be843 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:52:25.525089 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c115b2-6753-4247-8a4b-ebd3b90fb46f admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:52:25.535272 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c115b2-6753-4247-8a4b-ebd3b90fb46f admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:52:25.590514 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e12013-34af-4ce4-baa9-5fcf53c25823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:52:25.591294 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e12013-34af-4ce4-baa9-5fcf53c25823 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:52:25.591748 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e12013-34af-4ce4-baa9-5fcf53c25823 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:52:25.592497 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e12013-34af-4ce4-baa9-5fcf53c25823 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:52:25.593171 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e12013-34af-4ce4-baa9-5fcf53c25823 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:52:25.594959 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e12013-34af-4ce4-baa9-5fcf53c25823 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:52:25.604495 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1205/2420] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:52:25 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:52:25.608659 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4896d636-6861-4a29-bb99-27efec1d6bf3 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:52:25.624893 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4896d636-6861-4a29-bb99-27efec1d6bf3 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:52:25.629588 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44519265-c34b-429a-a3be-55e975cb2224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:52:25.630475 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44519265-c34b-429a-a3be-55e975cb2224 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:52:25.630753 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44519265-c34b-429a-a3be-55e975cb2224 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:52:25.631077 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44519265-c34b-429a-a3be-55e975cb2224 admin admin] PATH_INFO: `/v3/projects/0e9874be0cd542829242caded150fd52/users/020bbaed1f9148c4a9daa75180935b8e/roles/d89b6c92d35a401a8517f1d47f6be843` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:52:25.633069 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44519265-c34b-429a-a3be-55e975cb2224 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e9874be0cd542829242caded150fd52, user_id=020bbaed1f9148c4a9daa75180935b8e, role_id=d89b6c92d35a401a8517f1d47f6be843)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:52:25.635711 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44519265-c34b-429a-a3be-55e975cb2224 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:52:25.646739 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1216/2421] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:52:25 2022] PUT /identity/v3/projects/0e9874be0cd542829242caded150fd52/users/020bbaed1f9148c4a9daa75180935b8e/roles/d89b6c92d35a401a8517f1d47f6be843 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 22:52:28.537335 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e12013-34af-4ce4-baa9-5fcf53c25823 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:52:28.539650 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-a4e12013-34af-4ce4-baa9-5fcf53c25823 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:52:28.699596 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e12013-34af-4ce4-baa9-5fcf53c25823 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:52:28.716810 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:52:28.776020 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6921bfe2-d407-4ba0-872d-43ff926169b4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7829324db2574d38b18c749d0559177f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:52:28.776695 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6921bfe2-d407-4ba0-872d-43ff926169b4 service nova] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:52:28.776984 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6921bfe2-d407-4ba0-872d-43ff926169b4 service nova] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:52:28.777332 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6921bfe2-d407-4ba0-872d-43ff926169b4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:52:28.779375 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-6921bfe2-d407-4ba0-872d-43ff926169b4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:52:28.885436 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6921bfe2-d407-4ba0-872d-43ff926169b4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:52:28.887583 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6921bfe2-d407-4ba0-872d-43ff926169b4 service nova] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:52:28.900380 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1206/2422] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:52:28 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:52:32.029557 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a41f03-1dc8-40ef-8517-aa59a7b33697 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:52:32.030021 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a41f03-1dc8-40ef-8517-aa59a7b33697 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:52:32.030117 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a41f03-1dc8-40ef-8517-aa59a7b33697 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:52:32.062221 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-72a41f03-1dc8-40ef-8517-aa59a7b33697 None None] MFA Rules not processed for user `020bbaed1f9148c4a9daa75180935b8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:52:32.063525 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-72a41f03-1dc8-40ef-8517-aa59a7b33697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:52:32.185432 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1217/2423] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:52:32 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:52:32.203604 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6921bfe2-d407-4ba0-872d-43ff926169b4 service nova] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:52:32.225511 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6921bfe2-d407-4ba0-872d-43ff926169b4 service nova] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:52:32.232328 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c855eac-76be-45bf-ad56-2bf3a3846b9a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7829324db2574d38b18c749d0559177f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:52:32.235855 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c855eac-76be-45bf-ad56-2bf3a3846b9a service nova] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:52:32.236161 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c855eac-76be-45bf-ad56-2bf3a3846b9a service nova] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:52:32.236550 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c855eac-76be-45bf-ad56-2bf3a3846b9a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:52:32.317502 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c855eac-76be-45bf-ad56-2bf3a3846b9a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:52:32.319462 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c855eac-76be-45bf-ad56-2bf3a3846b9a service nova] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:52:32.331943 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1207/2424] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:52:32 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 22:52:35.647438 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc264ebb-2805-4a1c-bb65-093853cf7ad6 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:52:35.647970 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc264ebb-2805-4a1c-bb65-093853cf7ad6 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:52:35.648108 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc264ebb-2805-4a1c-bb65-093853cf7ad6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:52:35.668191 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-bc264ebb-2805-4a1c-bb65-093853cf7ad6 None None] MFA Rules not processed for user `3ce4c02d3e314761ba613a802194b709`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:52:35.669448 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-bc264ebb-2805-4a1c-bb65-093853cf7ad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:52:35.676542 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1218/2425] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:52:35 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:52:42.326481 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=76 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:52:45.660548 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=74,keystone:INSERT=5 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:52:46.653186 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659996c2-0049-490e-a332-b64977d5c94b None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:52:46.653695 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659996c2-0049-490e-a332-b64977d5c94b None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:52:46.653798 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659996c2-0049-490e-a332-b64977d5c94b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:52:46.672771 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-659996c2-0049-490e-a332-b64977d5c94b None None] MFA Rules not processed for user `3e072e59bf3e46aaa062ec85b0e2c91a`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:52:46.677886 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-659996c2-0049-490e-a332-b64977d5c94b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:52:46.680849 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1208/2426] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:52:46 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:52:56.666083 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=3 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:53:00.033179 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc264ebb-2805-4a1c-bb65-093853cf7ad6 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:53:00.088687 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc264ebb-2805-4a1c-bb65-093853cf7ad6 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:53:00.092514 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc63032-fde2-4c93-9325-1eae3e33ad40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:53:00.093186 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc63032-fde2-4c93-9325-1eae3e33ad40 service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:53:00.093475 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc63032-fde2-4c93-9325-1eae3e33ad40 service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:53:00.093755 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc63032-fde2-4c93-9325-1eae3e33ad40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:53:00.095600 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-ecc63032-fde2-4c93-9325-1eae3e33ad40 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:53:00.105807 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc63032-fde2-4c93-9325-1eae3e33ad40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:53:00.107171 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc63032-fde2-4c93-9325-1eae3e33ad40 service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:53:00.115406 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1219/2427] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:53:00 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:53:03.813796 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659996c2-0049-490e-a332-b64977d5c94b None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:53:03.829666 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659996c2-0049-490e-a332-b64977d5c94b None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:53:03.834688 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048b6de5-0db6-406e-8642-18d299122647 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7829324db2574d38b18c749d0559177f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:53:03.835380 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048b6de5-0db6-406e-8642-18d299122647 service nova] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:53:03.835665 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048b6de5-0db6-406e-8642-18d299122647 service nova] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:53:03.836028 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048b6de5-0db6-406e-8642-18d299122647 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:53:03.838134 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-048b6de5-0db6-406e-8642-18d299122647 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:53:03.850672 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048b6de5-0db6-406e-8642-18d299122647 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:53:03.852275 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048b6de5-0db6-406e-8642-18d299122647 service nova] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:53:03.863592 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1209/2428] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:53:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:53:08.015821 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc63032-fde2-4c93-9325-1eae3e33ad40 service neutron] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:53:08.025655 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc63032-fde2-4c93-9325-1eae3e33ad40 service neutron] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:53:08.029784 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c9b57c-0ef8-4e9e-a77f-9db17329e7f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:53:08.030569 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c9b57c-0ef8-4e9e-a77f-9db17329e7f5 service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:53:08.030958 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c9b57c-0ef8-4e9e-a77f-9db17329e7f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:53:08.031381 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c9b57c-0ef8-4e9e-a77f-9db17329e7f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:53:08.033303 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-80c9b57c-0ef8-4e9e-a77f-9db17329e7f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:53:08.111332 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c9b57c-0ef8-4e9e-a77f-9db17329e7f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:53:08.112963 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c9b57c-0ef8-4e9e-a77f-9db17329e7f5 service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:53:08.125936 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1220/2429] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:53:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 22:53:08.313891 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ca9467-883a-4965-b24c-61551b653fca None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:53:08.314430 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ca9467-883a-4965-b24c-61551b653fca None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:53:08.314430 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ca9467-883a-4965-b24c-61551b653fca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:53:08.332275 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-39ca9467-883a-4965-b24c-61551b653fca None None] MFA Rules not processed for user `0cf649be0c48492f8926f4436cbb3d6d`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:53:08.333485 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-39ca9467-883a-4965-b24c-61551b653fca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:53:08.389425 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1210/2430] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:53:08 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:53:14.004788 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c9b57c-0ef8-4e9e-a77f-9db17329e7f5 service neutron] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:53:14.006547 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-80c9b57c-0ef8-4e9e-a77f-9db17329e7f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:53:14.031235 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c9b57c-0ef8-4e9e-a77f-9db17329e7f5 service neutron] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:53:14.043407 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4553ec0-f3f8-4195-8d2a-e0e94c517c2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:53:14.044606 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4553ec0-f3f8-4195-8d2a-e0e94c517c2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:53:14.045050 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4553ec0-f3f8-4195-8d2a-e0e94c517c2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:53:14.045501 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4553ec0-f3f8-4195-8d2a-e0e94c517c2a admin admin] PATH_INFO: `/v3/users/1ed9249b2a9b42d9972ed71c04f2c888` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:53:14.047353 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4553ec0-f3f8-4195-8d2a-e0e94c517c2a admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ed9249b2a9b42d9972ed71c04f2c888)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:53:14.050010 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4553ec0-f3f8-4195-8d2a-e0e94c517c2a admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:53:14.339250 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-b4553ec0-f3f8-4195-8d2a-e0e94c517c2a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ed9249b2a9b42d9972ed71c04f2c888'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:53:14.339924 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-b4553ec0-f3f8-4195-8d2a-e0e94c517c2a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ed9249b2a9b42d9972ed71c04f2c888'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:53:14.340534 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-b4553ec0-f3f8-4195-8d2a-e0e94c517c2a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ed9249b2a9b42d9972ed71c04f2c888'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:53:14.348676 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-b4553ec0-f3f8-4195-8d2a-e0e94c517c2a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ed9249b2a9b42d9972ed71c04f2c888'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:53:14.426435 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-b4553ec0-f3f8-4195-8d2a-e0e94c517c2a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ed9249b2a9b42d9972ed71c04f2c888'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:53:14.444182 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1221/2431] 10.222.0.81 () {58 vars in 1223 bytes} [Wed Jul 20 22:53:14 2022] DELETE /identity/v3/users/1ed9249b2a9b42d9972ed71c04f2c888 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:53:14.699278 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ca9467-883a-4965-b24c-61551b653fca None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:53:14.711295 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ca9467-883a-4965-b24c-61551b653fca None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:53:14.768822 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38a108b-769e-4fc1-ac11-39da918ecdf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:53:14.769694 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38a108b-769e-4fc1-ac11-39da918ecdf9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:53:14.769978 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38a108b-769e-4fc1-ac11-39da918ecdf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:53:14.770256 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38a108b-769e-4fc1-ac11-39da918ecdf9 admin admin] PATH_INFO: `/v3/projects/55c688e93c2b414ca86a3c2f9ec84798` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:53:14.771483 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38a108b-769e-4fc1-ac11-39da918ecdf9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=55c688e93c2b414ca86a3c2f9ec84798)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:53:14.773410 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38a108b-769e-4fc1-ac11-39da918ecdf9 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:53:14.893264 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-a38a108b-769e-4fc1-ac11-39da918ecdf9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55c688e93c2b414ca86a3c2f9ec84798', 'request_id': 'req-a38a108b-769e-4fc1-ac11-39da918ecdf9'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:53:14.992204 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-a38a108b-769e-4fc1-ac11-39da918ecdf9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55c688e93c2b414ca86a3c2f9ec84798', 'request_id': 'req-a38a108b-769e-4fc1-ac11-39da918ecdf9'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:53:14.999383 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-a38a108b-769e-4fc1-ac11-39da918ecdf9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55c688e93c2b414ca86a3c2f9ec84798', 'request_id': 'req-a38a108b-769e-4fc1-ac11-39da918ecdf9'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:53:15.013560 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-a38a108b-769e-4fc1-ac11-39da918ecdf9 admin admin] The token cache is being invalidate because project 55c688e93c2b414ca86a3c2f9ec84798 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:53:15.013969 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-a38a108b-769e-4fc1-ac11-39da918ecdf9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:53:15.014979 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-a38a108b-769e-4fc1-ac11-39da918ecdf9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:53:15.016846 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1211/2432] 10.222.0.81 () {58 vars in 1232 bytes} [Wed Jul 20 22:53:14 2022] DELETE /identity/v3/projects/55c688e93c2b414ca86a3c2f9ec84798 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:53:15.022438 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e99f18-8280-4e6d-874f-355838185421 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:53:15.022925 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e99f18-8280-4e6d-874f-355838185421 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:53:15.023058 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e99f18-8280-4e6d-874f-355838185421 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:53:15.042394 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-69e99f18-8280-4e6d-874f-355838185421 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:53:15.043785 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-69e99f18-8280-4e6d-874f-355838185421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:53:15.048173 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:53:15.103748 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1222/2433] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:53:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:53:15.111754 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65509c2d-781e-42b4-84ff-e8c9ac7657ee None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:53:15.112040 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65509c2d-781e-42b4-84ff-e8c9ac7657ee None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:53:15.112318 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65509c2d-781e-42b4-84ff-e8c9ac7657ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:53:15.131102 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-65509c2d-781e-42b4-84ff-e8c9ac7657ee None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:53:15.133165 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-65509c2d-781e-42b4-84ff-e8c9ac7657ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:53:15.142844 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1212/2434] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:53:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:53:15.147507 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e99f18-8280-4e6d-874f-355838185421 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:53:15.157866 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e99f18-8280-4e6d-874f-355838185421 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:53:15.163551 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db4723a-8d9d-4971-abbd-a096bd5f9ec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:53:15.164231 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db4723a-8d9d-4971-abbd-a096bd5f9ec2 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:53:15.164501 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db4723a-8d9d-4971-abbd-a096bd5f9ec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:53:15.164781 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db4723a-8d9d-4971-abbd-a096bd5f9ec2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:53:15.166073 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db4723a-8d9d-4971-abbd-a096bd5f9ec2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:53:15.166382 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db4723a-8d9d-4971-abbd-a096bd5f9ec2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:53:15.168083 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db4723a-8d9d-4971-abbd-a096bd5f9ec2 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:53:15.180197 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1223/2435] 10.222.0.81 () {58 vars in 1152 bytes} [Wed Jul 20 22:53:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:53:15.185023 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65509c2d-781e-42b4-84ff-e8c9ac7657ee None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:53:15.194933 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65509c2d-781e-42b4-84ff-e8c9ac7657ee None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:53:15.202408 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390f3965-f213-4d67-a63b-ab057050d306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:53:15.202408 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390f3965-f213-4d67-a63b-ab057050d306 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:53:15.202408 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390f3965-f213-4d67-a63b-ab057050d306 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:53:15.202408 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390f3965-f213-4d67-a63b-ab057050d306 admin admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:53:15.202863 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390f3965-f213-4d67-a63b-ab057050d306 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:53:15.204330 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390f3965-f213-4d67-a63b-ab057050d306 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:53:15.217960 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:53:15.268612 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1213/2436] 10.222.0.81 () {60 vars in 1152 bytes} [Wed Jul 20 22:53:15 2022] POST /identity/v3/projects => generated 387 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:53:15.273377 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db4723a-8d9d-4971-abbd-a096bd5f9ec2 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:53:15.283820 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db4723a-8d9d-4971-abbd-a096bd5f9ec2 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:53:15.346982 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4712042a-6f13-4812-9fe9-89d9e13e681b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:53:15.347773 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4712042a-6f13-4812-9fe9-89d9e13e681b admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:53:15.348066 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4712042a-6f13-4812-9fe9-89d9e13e681b admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:53:15.348349 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4712042a-6f13-4812-9fe9-89d9e13e681b admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:53:15.349083 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4712042a-6f13-4812-9fe9-89d9e13e681b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:53:15.351046 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4712042a-6f13-4812-9fe9-89d9e13e681b admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:53:15.488127 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1224/2437] 10.222.0.81 () {60 vars in 1143 bytes} [Wed Jul 20 22:53:15 2022] POST /identity/v3/users => generated 294 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:53:15.489526 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390f3965-f213-4d67-a63b-ab057050d306 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:53:15.507422 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390f3965-f213-4d67-a63b-ab057050d306 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:53:15.514475 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f0bf6f-8253-48fe-8d13-55081dd72861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:53:15.515617 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f0bf6f-8253-48fe-8d13-55081dd72861 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:53:15.515913 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f0bf6f-8253-48fe-8d13-55081dd72861 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:53:15.516244 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f0bf6f-8253-48fe-8d13-55081dd72861 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:53:15.517007 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f0bf6f-8253-48fe-8d13-55081dd72861 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:53:15.518608 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f0bf6f-8253-48fe-8d13-55081dd72861 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:53:15.530538 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1214/2438] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:53:15 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 22:53:15.534539 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4712042a-6f13-4812-9fe9-89d9e13e681b admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:53:15.546051 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4712042a-6f13-4812-9fe9-89d9e13e681b admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:53:15.551039 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dda1d43-702d-44e4-848a-d8fedaf7f43f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:53:15.557189 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dda1d43-702d-44e4-848a-d8fedaf7f43f admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:53:15.557189 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dda1d43-702d-44e4-848a-d8fedaf7f43f admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:53:15.557189 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dda1d43-702d-44e4-848a-d8fedaf7f43f admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:53:15.557189 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dda1d43-702d-44e4-848a-d8fedaf7f43f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:53:15.557189 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dda1d43-702d-44e4-848a-d8fedaf7f43f admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:53:15.565638 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1225/2439] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:53:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:53:15.569997 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f0bf6f-8253-48fe-8d13-55081dd72861 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:53:15.579978 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f0bf6f-8253-48fe-8d13-55081dd72861 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:53:15.584539 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6578c7-4b44-4313-ad1b-432689325e41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:53:15.585454 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6578c7-4b44-4313-ad1b-432689325e41 admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:53:15.585733 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6578c7-4b44-4313-ad1b-432689325e41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:53:15.586009 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6578c7-4b44-4313-ad1b-432689325e41 admin admin] PATH_INFO: `/v3/projects/7ace18b7cfda4329adb560220ff909cc/users/bc19753d420840ffa5cde19d4c1fac8f/roles/559942c4de1744169f8b5e0404ecde6b` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:53:15.601534 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6578c7-4b44-4313-ad1b-432689325e41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ace18b7cfda4329adb560220ff909cc, user_id=bc19753d420840ffa5cde19d4c1fac8f, role_id=559942c4de1744169f8b5e0404ecde6b)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:53:15.604850 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6578c7-4b44-4313-ad1b-432689325e41 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:53:15.700894 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1215/2440] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:53:15 2022] PUT /identity/v3/projects/7ace18b7cfda4329adb560220ff909cc/users/bc19753d420840ffa5cde19d4c1fac8f/roles/559942c4de1744169f8b5e0404ecde6b => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:53:15.706091 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9096448e-5c2d-408b-b54a-5ca89f95817c None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:53:15.706381 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9096448e-5c2d-408b-b54a-5ca89f95817c None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:53:15.706661 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9096448e-5c2d-408b-b54a-5ca89f95817c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:53:15.727313 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-9096448e-5c2d-408b-b54a-5ca89f95817c None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:53:15.729147 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-9096448e-5c2d-408b-b54a-5ca89f95817c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:53:15.789387 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1226/2441] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:53:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:53:15.799803 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6578c7-4b44-4313-ad1b-432689325e41 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:53:15.801539 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-5b6578c7-4b44-4313-ad1b-432689325e41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:53:15.858806 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6578c7-4b44-4313-ad1b-432689325e41 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:53:15.862571 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f5f7b9-fa9c-44e2-88cc-33acd1062cab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:53:15.863249 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f5f7b9-fa9c-44e2-88cc-33acd1062cab service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:53:15.863571 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f5f7b9-fa9c-44e2-88cc-33acd1062cab service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:53:15.863850 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f5f7b9-fa9c-44e2-88cc-33acd1062cab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:53:15.872988 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f5f7b9-fa9c-44e2-88cc-33acd1062cab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:53:15.874426 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f5f7b9-fa9c-44e2-88cc-33acd1062cab service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:53:15.884469 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1216/2442] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:53:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:53:20.980876 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9096448e-5c2d-408b-b54a-5ca89f95817c None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:53:21.001431 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:53:21.003624 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9096448e-5c2d-408b-b54a-5ca89f95817c None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:53:21.007778 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc609d3-fe23-4c3d-9a6c-47b695a1229a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:53:21.008052 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc609d3-fe23-4c3d-9a6c-47b695a1229a service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:53:21.008505 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc609d3-fe23-4c3d-9a6c-47b695a1229a service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:53:21.009754 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc609d3-fe23-4c3d-9a6c-47b695a1229a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:53:21.011547 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-7bc609d3-fe23-4c3d-9a6c-47b695a1229a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:53:21.128632 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc609d3-fe23-4c3d-9a6c-47b695a1229a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:53:21.130138 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc609d3-fe23-4c3d-9a6c-47b695a1229a service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:53:21.141743 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1227/2443] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:53:20 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:53:22.429757 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9514d9fc-2d48-4f82-9d35-842a0313ee43 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:53:22.431281 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9514d9fc-2d48-4f82-9d35-842a0313ee43 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:53:22.431825 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9514d9fc-2d48-4f82-9d35-842a0313ee43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:53:22.478428 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-9514d9fc-2d48-4f82-9d35-842a0313ee43 None None] MFA Rules not processed for user `bc19753d420840ffa5cde19d4c1fac8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:53:22.483136 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-9514d9fc-2d48-4f82-9d35-842a0313ee43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:53:22.569017 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1217/2444] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:53:22 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:53:22.585663 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc609d3-fe23-4c3d-9a6c-47b695a1229a service neutron] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:53:22.587386 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-7bc609d3-fe23-4c3d-9a6c-47b695a1229a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:53:22.741719 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc609d3-fe23-4c3d-9a6c-47b695a1229a service neutron] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:53:22.750440 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6145d43d-f2ec-4833-a335-d8d380ffa09d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7829324db2574d38b18c749d0559177f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:53:22.755594 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6145d43d-f2ec-4833-a335-d8d380ffa09d service nova] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:53:22.755594 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6145d43d-f2ec-4833-a335-d8d380ffa09d service nova] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:53:22.755594 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6145d43d-f2ec-4833-a335-d8d380ffa09d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:53:22.766074 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6145d43d-f2ec-4833-a335-d8d380ffa09d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:53:22.768176 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6145d43d-f2ec-4833-a335-d8d380ffa09d service nova] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:53:22.787232 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1228/2445] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:53:22 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:53:26.252091 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f8fa49-22b8-4512-9855-f2072565f865 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:53:26.252600 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f8fa49-22b8-4512-9855-f2072565f865 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:53:26.252745 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f8fa49-22b8-4512-9855-f2072565f865 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:53:26.272211 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-d6f8fa49-22b8-4512-9855-f2072565f865 None None] MFA Rules not processed for user `bc19753d420840ffa5cde19d4c1fac8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:53:26.282356 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-d6f8fa49-22b8-4512-9855-f2072565f865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:53:26.282356 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1218/2446] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:53:26 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:53:32.774569 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=91 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:53:36.266860 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:INSERT=2,keystone:SELECT=78 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:54:26.044582 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6145d43d-f2ec-4833-a335-d8d380ffa09d service nova] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:54:26.046121 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-6145d43d-f2ec-4833-a335-d8d380ffa09d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:54:26.135320 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6145d43d-f2ec-4833-a335-d8d380ffa09d service nova] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:54:26.140604 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810f6b5d-18c3-4113-9b03-8416517ea1fd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5500cebdd9d4f8e9aecb45d6c955878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:54:26.141264 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810f6b5d-18c3-4113-9b03-8416517ea1fd service placement] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:54:26.141553 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810f6b5d-18c3-4113-9b03-8416517ea1fd service placement] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:54:26.141837 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810f6b5d-18c3-4113-9b03-8416517ea1fd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:54:26.143624 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-810f6b5d-18c3-4113-9b03-8416517ea1fd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:54:26.155864 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810f6b5d-18c3-4113-9b03-8416517ea1fd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:54:26.157490 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810f6b5d-18c3-4113-9b03-8416517ea1fd service placement] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:54:26.168115 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1229/2447] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:54:26 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:54:36.163121 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=41 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:54:47.822805 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7511f22b-40fa-409d-8389-6d80584b7e0f None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:54:47.823438 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7511f22b-40fa-409d-8389-6d80584b7e0f None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:54:47.823438 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7511f22b-40fa-409d-8389-6d80584b7e0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:54:47.842977 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-7511f22b-40fa-409d-8389-6d80584b7e0f None None] MFA Rules not processed for user `3e072e59bf3e46aaa062ec85b0e2c91a`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:54:47.844220 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-7511f22b-40fa-409d-8389-6d80584b7e0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:54:47.900009 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1219/2448] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:54:47 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:54:57.892554 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=25 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:55:25.112112 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810f6b5d-18c3-4113-9b03-8416517ea1fd service placement] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:55:25.130371 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810f6b5d-18c3-4113-9b03-8416517ea1fd service placement] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:55:25.134261 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2318eae-c609-47af-a9ec-e5db78dcc8a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:55:25.134974 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2318eae-c609-47af-a9ec-e5db78dcc8a5 service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:55:25.135289 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2318eae-c609-47af-a9ec-e5db78dcc8a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:55:25.135568 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2318eae-c609-47af-a9ec-e5db78dcc8a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:55:25.137465 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-e2318eae-c609-47af-a9ec-e5db78dcc8a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:55:25.151279 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2318eae-c609-47af-a9ec-e5db78dcc8a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:55:25.152700 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2318eae-c609-47af-a9ec-e5db78dcc8a5 service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:55:25.164288 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1230/2449] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:55:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 22:55:25.410835 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7511f22b-40fa-409d-8389-6d80584b7e0f None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:55:25.412460 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-7511f22b-40fa-409d-8389-6d80584b7e0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:55:25.437672 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7511f22b-40fa-409d-8389-6d80584b7e0f None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:55:25.441233 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060395e3-3227-4db1-a029-95535119c410 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:55:25.441890 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060395e3-3227-4db1-a029-95535119c410 service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:55:25.442174 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060395e3-3227-4db1-a029-95535119c410 service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:55:25.442459 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060395e3-3227-4db1-a029-95535119c410 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:55:25.444251 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-060395e3-3227-4db1-a029-95535119c410 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:55:25.458611 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060395e3-3227-4db1-a029-95535119c410 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:55:25.460081 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060395e3-3227-4db1-a029-95535119c410 service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:55:25.480613 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1220/2450] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:55:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:55:32.627456 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b505a505-265b-4741-890f-e183b5a04f43 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:55:32.631492 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b505a505-265b-4741-890f-e183b5a04f43 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:55:32.631492 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b505a505-265b-4741-890f-e183b5a04f43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:55:32.655443 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-b505a505-265b-4741-890f-e183b5a04f43 None None] MFA Rules not processed for user `90915e1264324f16bc7bae7e46e0c97b`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:55:32.656657 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-b505a505-265b-4741-890f-e183b5a04f43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:55:32.720004 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1231/2451] 10.222.0.81 () {58 vars in 1009 bytes} [Wed Jul 20 22:55:32 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:55:32.753865 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060395e3-3227-4db1-a029-95535119c410 service neutron] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:55:32.762786 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060395e3-3227-4db1-a029-95535119c410 service neutron] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:55:32.763362 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b505a505-265b-4741-890f-e183b5a04f43 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:55:32.764728 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-b505a505-265b-4741-890f-e183b5a04f43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:55:32.766507 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b327ddf-b123-4fef-ade2-b9cde37380f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:55:32.767197 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b327ddf-b123-4fef-ade2-b9cde37380f8 service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:55:32.767493 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b327ddf-b123-4fef-ade2-b9cde37380f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:55:32.767776 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b327ddf-b123-4fef-ade2-b9cde37380f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:55:32.769521 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-8b327ddf-b123-4fef-ade2-b9cde37380f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:55:32.804565 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b327ddf-b123-4fef-ade2-b9cde37380f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:55:32.805998 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b327ddf-b123-4fef-ade2-b9cde37380f8 service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:55:32.824927 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1221/2452] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:55:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:55:32.835087 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b505a505-265b-4741-890f-e183b5a04f43 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:55:32.840251 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5888ca1e-1d28-4717-8b79-cd1e7e7a83e1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70e97436a08543a180225561ef8bc817', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:55:32.840908 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5888ca1e-1d28-4717-8b79-cd1e7e7a83e1 service swift] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:55:32.841752 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5888ca1e-1d28-4717-8b79-cd1e7e7a83e1 service swift] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:55:32.842032 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5888ca1e-1d28-4717-8b79-cd1e7e7a83e1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:55:32.850708 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5888ca1e-1d28-4717-8b79-cd1e7e7a83e1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:55:32.852328 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5888ca1e-1d28-4717-8b79-cd1e7e7a83e1 service swift] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:55:32.860662 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1232/2453] 10.222.0.81 () {60 vars in 1384 bytes} [Wed Jul 20 22:55:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 20 22:55:38.248227 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b327ddf-b123-4fef-ade2-b9cde37380f8 service neutron] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:55:38.266722 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b327ddf-b123-4fef-ade2-b9cde37380f8 service neutron] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:55:38.272045 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44569e55-f586-4083-8964-5b0c4daae2ec service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5500cebdd9d4f8e9aecb45d6c955878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:55:38.272826 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44569e55-f586-4083-8964-5b0c4daae2ec service placement] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:55:38.273073 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44569e55-f586-4083-8964-5b0c4daae2ec service placement] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:55:38.273442 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44569e55-f586-4083-8964-5b0c4daae2ec service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:55:38.275366 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-44569e55-f586-4083-8964-5b0c4daae2ec service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:55:38.290278 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44569e55-f586-4083-8964-5b0c4daae2ec service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:55:38.292152 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44569e55-f586-4083-8964-5b0c4daae2ec service placement] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:55:38.306329 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1222/2454] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:55:38 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:55:38.404447 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5888ca1e-1d28-4717-8b79-cd1e7e7a83e1 service swift] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:55:38.405942 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-5888ca1e-1d28-4717-8b79-cd1e7e7a83e1 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:55:38.432562 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5888ca1e-1d28-4717-8b79-cd1e7e7a83e1 service swift] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:55:38.442763 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a5bc83-f3ba-489a-a0f8-0121a1339d7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:55:38.443719 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a5bc83-f3ba-489a-a0f8-0121a1339d7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:55:38.444021 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a5bc83-f3ba-489a-a0f8-0121a1339d7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:55:38.444308 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a5bc83-f3ba-489a-a0f8-0121a1339d7e admin admin] PATH_INFO: `/v3/users/0cf649be0c48492f8926f4436cbb3d6d` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:55:38.462994 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a5bc83-f3ba-489a-a0f8-0121a1339d7e admin admin] RBAC: Authorizing `identity:delete_user(user_id=0cf649be0c48492f8926f4436cbb3d6d)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:55:38.465224 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a5bc83-f3ba-489a-a0f8-0121a1339d7e admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:55:38.609666 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-77a5bc83-f3ba-489a-a0f8-0121a1339d7e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0cf649be0c48492f8926f4436cbb3d6d'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:55:38.610674 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-77a5bc83-f3ba-489a-a0f8-0121a1339d7e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0cf649be0c48492f8926f4436cbb3d6d'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:55:38.611381 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-77a5bc83-f3ba-489a-a0f8-0121a1339d7e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0cf649be0c48492f8926f4436cbb3d6d'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:55:38.612787 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:55:38.626236 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-77a5bc83-f3ba-489a-a0f8-0121a1339d7e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0cf649be0c48492f8926f4436cbb3d6d'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:55:38.688358 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-77a5bc83-f3ba-489a-a0f8-0121a1339d7e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0cf649be0c48492f8926f4436cbb3d6d'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:55:38.709311 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1233/2455] 10.222.0.81 () {58 vars in 1223 bytes} [Wed Jul 20 22:55:38 2022] DELETE /identity/v3/users/0cf649be0c48492f8926f4436cbb3d6d => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:55:38.970430 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44569e55-f586-4083-8964-5b0c4daae2ec service placement] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:55:38.986538 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44569e55-f586-4083-8964-5b0c4daae2ec service placement] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:55:39.054448 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb24ed00-be35-4f48-b9ae-dad23394e544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:55:39.055366 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb24ed00-be35-4f48-b9ae-dad23394e544 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:55:39.055649 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb24ed00-be35-4f48-b9ae-dad23394e544 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:55:39.055926 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb24ed00-be35-4f48-b9ae-dad23394e544 admin admin] PATH_INFO: `/v3/projects/90ac023afcb44b5097539298076c0838` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:55:39.068539 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb24ed00-be35-4f48-b9ae-dad23394e544 admin admin] RBAC: Authorizing `identity:delete_project(project_id=90ac023afcb44b5097539298076c0838)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:55:39.070330 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb24ed00-be35-4f48-b9ae-dad23394e544 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:55:39.203787 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-cb24ed00-be35-4f48-b9ae-dad23394e544 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90ac023afcb44b5097539298076c0838', 'request_id': 'req-cb24ed00-be35-4f48-b9ae-dad23394e544'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:55:39.288288 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-cb24ed00-be35-4f48-b9ae-dad23394e544 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90ac023afcb44b5097539298076c0838', 'request_id': 'req-cb24ed00-be35-4f48-b9ae-dad23394e544'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:55:39.294561 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-cb24ed00-be35-4f48-b9ae-dad23394e544 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90ac023afcb44b5097539298076c0838', 'request_id': 'req-cb24ed00-be35-4f48-b9ae-dad23394e544'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:55:39.307275 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-cb24ed00-be35-4f48-b9ae-dad23394e544 admin admin] The token cache is being invalidate because project 90ac023afcb44b5097539298076c0838 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:55:39.307599 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-cb24ed00-be35-4f48-b9ae-dad23394e544 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:55:39.308471 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-cb24ed00-be35-4f48-b9ae-dad23394e544 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:55:39.310294 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1223/2456] 10.222.0.81 () {58 vars in 1232 bytes} [Wed Jul 20 22:55:38 2022] DELETE /identity/v3/projects/90ac023afcb44b5097539298076c0838 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:55:39.315446 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6a922a-5785-4724-a4c8-2063ef803867 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:55:39.315736 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6a922a-5785-4724-a4c8-2063ef803867 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:55:39.316017 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6a922a-5785-4724-a4c8-2063ef803867 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:55:39.387721 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-df6a922a-5785-4724-a4c8-2063ef803867 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:55:39.389524 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-df6a922a-5785-4724-a4c8-2063ef803867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:55:39.457099 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1234/2457] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:55:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:55:39.466929 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889c6d6b-dc20-4e2c-9cc2-dc132282b43f None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:55:39.467483 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889c6d6b-dc20-4e2c-9cc2-dc132282b43f None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:55:39.467770 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889c6d6b-dc20-4e2c-9cc2-dc132282b43f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:55:39.474949 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:55:39.488625 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-889c6d6b-dc20-4e2c-9cc2-dc132282b43f None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:55:39.489882 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-889c6d6b-dc20-4e2c-9cc2-dc132282b43f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:55:39.498718 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1224/2458] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:55:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:55:39.503891 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6a922a-5785-4724-a4c8-2063ef803867 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:55:39.516353 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6a922a-5785-4724-a4c8-2063ef803867 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:55:39.521765 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174a7793-8379-4c82-b971-56a148ab263f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:55:39.522462 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174a7793-8379-4c82-b971-56a148ab263f admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:55:39.522748 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174a7793-8379-4c82-b971-56a148ab263f admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:55:39.523124 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174a7793-8379-4c82-b971-56a148ab263f admin admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:55:39.523817 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174a7793-8379-4c82-b971-56a148ab263f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:55:39.530208 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174a7793-8379-4c82-b971-56a148ab263f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:55:39.530208 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174a7793-8379-4c82-b971-56a148ab263f admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:55:39.540298 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1235/2459] 10.222.0.81 () {58 vars in 1152 bytes} [Wed Jul 20 22:55:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:55:39.544705 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889c6d6b-dc20-4e2c-9cc2-dc132282b43f None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:55:39.555197 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889c6d6b-dc20-4e2c-9cc2-dc132282b43f None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:55:39.560376 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2150eeec-90d4-45e3-93ce-aff01cbd7e07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:55:39.561040 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2150eeec-90d4-45e3-93ce-aff01cbd7e07 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:55:39.561347 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2150eeec-90d4-45e3-93ce-aff01cbd7e07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:55:39.561620 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2150eeec-90d4-45e3-93ce-aff01cbd7e07 admin admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:55:39.562305 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2150eeec-90d4-45e3-93ce-aff01cbd7e07 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:55:39.564243 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2150eeec-90d4-45e3-93ce-aff01cbd7e07 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:55:40.112734 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1225/2460] 10.222.0.81 () {60 vars in 1152 bytes} [Wed Jul 20 22:55:39 2022] POST /identity/v3/projects => generated 373 bytes in 569 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:55:40.117095 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174a7793-8379-4c82-b971-56a148ab263f admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:55:40.127909 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174a7793-8379-4c82-b971-56a148ab263f admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:55:40.195104 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba0f384-0786-4380-aae8-69505a12f412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:55:40.195104 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba0f384-0786-4380-aae8-69505a12f412 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:55:40.195104 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba0f384-0786-4380-aae8-69505a12f412 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:55:40.195104 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba0f384-0786-4380-aae8-69505a12f412 admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:55:40.195616 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba0f384-0786-4380-aae8-69505a12f412 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:55:40.195616 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba0f384-0786-4380-aae8-69505a12f412 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:55:40.278012 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1236/2461] 10.222.0.81 () {60 vars in 1143 bytes} [Wed Jul 20 22:55:40 2022] POST /identity/v3/users => generated 287 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:55:40.283071 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2150eeec-90d4-45e3-93ce-aff01cbd7e07 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:55:40.295043 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2150eeec-90d4-45e3-93ce-aff01cbd7e07 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:55:40.300183 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2142cca-f7c3-4fbe-8148-6c75b54cbab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:55:40.301056 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2142cca-f7c3-4fbe-8148-6c75b54cbab8 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:55:40.301217 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2142cca-f7c3-4fbe-8148-6c75b54cbab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:55:40.301574 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2142cca-f7c3-4fbe-8148-6c75b54cbab8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:55:40.302553 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2142cca-f7c3-4fbe-8148-6c75b54cbab8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:55:40.304277 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2142cca-f7c3-4fbe-8148-6c75b54cbab8 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:55:40.316579 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1226/2462] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:55:40 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 22:55:40.320971 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba0f384-0786-4380-aae8-69505a12f412 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:55:40.333309 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba0f384-0786-4380-aae8-69505a12f412 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:55:40.338295 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d92da12-9bd7-4a48-803f-48459c8c5c96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:55:40.338972 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d92da12-9bd7-4a48-803f-48459c8c5c96 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:55:40.339292 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d92da12-9bd7-4a48-803f-48459c8c5c96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:55:40.339582 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d92da12-9bd7-4a48-803f-48459c8c5c96 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:55:40.340315 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d92da12-9bd7-4a48-803f-48459c8c5c96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:55:40.341911 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d92da12-9bd7-4a48-803f-48459c8c5c96 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:55:40.348342 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:55:40.355511 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1237/2463] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:55:40 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:55:40.360105 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2142cca-f7c3-4fbe-8148-6c75b54cbab8 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:55:40.371072 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2142cca-f7c3-4fbe-8148-6c75b54cbab8 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:55:40.377088 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dea5ab-1f83-4f16-b41c-80510c1061ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:55:40.378157 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dea5ab-1f83-4f16-b41c-80510c1061ef admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:55:40.378549 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dea5ab-1f83-4f16-b41c-80510c1061ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:55:40.378932 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dea5ab-1f83-4f16-b41c-80510c1061ef admin admin] PATH_INFO: `/v3/projects/be2475b9385d4c9c9377cec5b1c18c22/users/af8afe42fff94ce69070530e4032d3e0/roles/559942c4de1744169f8b5e0404ecde6b` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:55:40.396329 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dea5ab-1f83-4f16-b41c-80510c1061ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=be2475b9385d4c9c9377cec5b1c18c22, user_id=af8afe42fff94ce69070530e4032d3e0, role_id=559942c4de1744169f8b5e0404ecde6b)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:55:40.399374 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dea5ab-1f83-4f16-b41c-80510c1061ef admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:55:40.516302 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552a4cc5-244f-4aed-88b0-43248515ad03 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:55:40.516302 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552a4cc5-244f-4aed-88b0-43248515ad03 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:55:40.516302 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552a4cc5-244f-4aed-88b0-43248515ad03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:55:40.517218 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1227/2464] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:55:40 2022] PUT /identity/v3/projects/be2475b9385d4c9c9377cec5b1c18c22/users/af8afe42fff94ce69070530e4032d3e0/roles/559942c4de1744169f8b5e0404ecde6b => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:55:40.545576 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-552a4cc5-244f-4aed-88b0-43248515ad03 None None] MFA Rules not processed for user `af8afe42fff94ce69070530e4032d3e0`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:55:40.546900 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-552a4cc5-244f-4aed-88b0-43248515ad03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:55:40.617256 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1238/2465] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:55:40 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:55:40.628785 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dea5ab-1f83-4f16-b41c-80510c1061ef admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:55:40.630958 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-e4dea5ab-1f83-4f16-b41c-80510c1061ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:55:40.715533 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dea5ab-1f83-4f16-b41c-80510c1061ef admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:55:40.723361 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b4bfec-59c4-4f2b-b3d1-52f26d306ce4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7829324db2574d38b18c749d0559177f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:55:40.724434 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b4bfec-59c4-4f2b-b3d1-52f26d306ce4 service nova] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:55:40.724767 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b4bfec-59c4-4f2b-b3d1-52f26d306ce4 service nova] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:55:40.725055 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b4bfec-59c4-4f2b-b3d1-52f26d306ce4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:55:40.734048 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b4bfec-59c4-4f2b-b3d1-52f26d306ce4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:55:40.735763 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b4bfec-59c4-4f2b-b3d1-52f26d306ce4 service nova] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:55:40.755200 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1228/2466] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:55:40 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:55:41.030179 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552a4cc5-244f-4aed-88b0-43248515ad03 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:55:41.101423 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552a4cc5-244f-4aed-88b0-43248515ad03 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:55:41.105774 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d1db49-533d-48f9-9354-a4a12c20594a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:55:41.106680 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d1db49-533d-48f9-9354-a4a12c20594a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:55:41.106993 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d1db49-533d-48f9-9354-a4a12c20594a admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:55:41.107349 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d1db49-533d-48f9-9354-a4a12c20594a admin admin] PATH_INFO: `/v3/users/af8afe42fff94ce69070530e4032d3e0` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:55:41.108702 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d1db49-533d-48f9-9354-a4a12c20594a admin admin] RBAC: Authorizing `identity:delete_user(user_id=af8afe42fff94ce69070530e4032d3e0)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:55:41.110642 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d1db49-533d-48f9-9354-a4a12c20594a admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:55:41.303352 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-f0d1db49-533d-48f9-9354-a4a12c20594a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af8afe42fff94ce69070530e4032d3e0'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:55:41.304459 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-f0d1db49-533d-48f9-9354-a4a12c20594a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af8afe42fff94ce69070530e4032d3e0'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:55:41.305395 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-f0d1db49-533d-48f9-9354-a4a12c20594a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af8afe42fff94ce69070530e4032d3e0'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:55:41.316167 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-f0d1db49-533d-48f9-9354-a4a12c20594a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af8afe42fff94ce69070530e4032d3e0'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:55:41.396677 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-f0d1db49-533d-48f9-9354-a4a12c20594a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af8afe42fff94ce69070530e4032d3e0'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:55:41.416766 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1239/2467] 10.222.0.81 () {58 vars in 1223 bytes} [Wed Jul 20 22:55:41 2022] DELETE /identity/v3/users/af8afe42fff94ce69070530e4032d3e0 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:55:41.421890 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b62e1c-a61f-463f-a406-a123f1a9bc86 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:55:41.422376 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b62e1c-a61f-463f-a406-a123f1a9bc86 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:55:41.422483 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b62e1c-a61f-463f-a406-a123f1a9bc86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:55:41.442526 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-d9b62e1c-a61f-463f-a406-a123f1a9bc86 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:55:41.443772 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-d9b62e1c-a61f-463f-a406-a123f1a9bc86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:55:41.514176 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1229/2468] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:55:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:55:41.524987 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d1db49-533d-48f9-9354-a4a12c20594a admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:55:41.525671 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-f0d1db49-533d-48f9-9354-a4a12c20594a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:55:41.536968 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:55:41.587019 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d1db49-533d-48f9-9354-a4a12c20594a admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:55:41.590864 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c27ae3c-9f9a-47b5-a384-f20293df5a13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:55:41.591713 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c27ae3c-9f9a-47b5-a384-f20293df5a13 service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:55:41.591997 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c27ae3c-9f9a-47b5-a384-f20293df5a13 service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:55:41.592274 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c27ae3c-9f9a-47b5-a384-f20293df5a13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:55:41.601792 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c27ae3c-9f9a-47b5-a384-f20293df5a13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:55:41.603672 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c27ae3c-9f9a-47b5-a384-f20293df5a13 service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:55:41.614026 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1240/2469] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:55:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:55:42.056872 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b62e1c-a61f-463f-a406-a123f1a9bc86 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:55:42.056872 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-d9b62e1c-a61f-463f-a406-a123f1a9bc86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:55:42.070928 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b62e1c-a61f-463f-a406-a123f1a9bc86 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:55:42.077249 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a47226-e65c-489e-8e57-7b8b03d00fef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:55:42.078335 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a47226-e65c-489e-8e57-7b8b03d00fef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:55:42.078799 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a47226-e65c-489e-8e57-7b8b03d00fef admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:55:42.079255 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a47226-e65c-489e-8e57-7b8b03d00fef admin admin] PATH_INFO: `/v3/projects/be2475b9385d4c9c9377cec5b1c18c22` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:55:42.080726 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a47226-e65c-489e-8e57-7b8b03d00fef admin admin] RBAC: Authorizing `identity:delete_project(project_id=be2475b9385d4c9c9377cec5b1c18c22)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:55:42.083751 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a47226-e65c-489e-8e57-7b8b03d00fef admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:55:42.101744 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:55:42.237808 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-f7a47226-e65c-489e-8e57-7b8b03d00fef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be2475b9385d4c9c9377cec5b1c18c22', 'request_id': 'req-f7a47226-e65c-489e-8e57-7b8b03d00fef'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:55:42.311629 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-f7a47226-e65c-489e-8e57-7b8b03d00fef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be2475b9385d4c9c9377cec5b1c18c22', 'request_id': 'req-f7a47226-e65c-489e-8e57-7b8b03d00fef'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:55:42.321292 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-f7a47226-e65c-489e-8e57-7b8b03d00fef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be2475b9385d4c9c9377cec5b1c18c22', 'request_id': 'req-f7a47226-e65c-489e-8e57-7b8b03d00fef'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:55:42.340689 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-f7a47226-e65c-489e-8e57-7b8b03d00fef admin admin] The token cache is being invalidate because project be2475b9385d4c9c9377cec5b1c18c22 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:55:42.341175 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-f7a47226-e65c-489e-8e57-7b8b03d00fef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:55:42.341937 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-f7a47226-e65c-489e-8e57-7b8b03d00fef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:55:42.344195 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1230/2470] 10.222.0.81 () {58 vars in 1232 bytes} [Wed Jul 20 22:55:42 2022] DELETE /identity/v3/projects/be2475b9385d4c9c9377cec5b1c18c22 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:55:42.350239 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72daff86-56b9-4fff-8db0-93ac420367a0 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:55:42.350795 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72daff86-56b9-4fff-8db0-93ac420367a0 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:55:42.352213 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72daff86-56b9-4fff-8db0-93ac420367a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:55:42.373830 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-72daff86-56b9-4fff-8db0-93ac420367a0 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:55:42.374971 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-72daff86-56b9-4fff-8db0-93ac420367a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:55:42.456469 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1241/2471] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:55:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:55:42.466858 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f4f6b3-068a-4b8d-9bdb-b07c15212c56 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:55:42.467420 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f4f6b3-068a-4b8d-9bdb-b07c15212c56 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:55:42.467792 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f4f6b3-068a-4b8d-9bdb-b07c15212c56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:55:42.487479 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-c9f4f6b3-068a-4b8d-9bdb-b07c15212c56 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:55:42.488876 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-c9f4f6b3-068a-4b8d-9bdb-b07c15212c56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:55:42.498096 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1231/2472] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:55:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:55:42.502319 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72daff86-56b9-4fff-8db0-93ac420367a0 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:55:42.516002 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72daff86-56b9-4fff-8db0-93ac420367a0 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:55:42.520935 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca044f78-b6e8-4f40-88b6-986135d9987c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:55:42.521604 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca044f78-b6e8-4f40-88b6-986135d9987c admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:55:42.521892 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca044f78-b6e8-4f40-88b6-986135d9987c admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:55:42.522171 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca044f78-b6e8-4f40-88b6-986135d9987c admin admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:55:42.522902 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca044f78-b6e8-4f40-88b6-986135d9987c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:55:42.523243 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca044f78-b6e8-4f40-88b6-986135d9987c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:55:42.524914 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca044f78-b6e8-4f40-88b6-986135d9987c admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:55:42.552523 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f4f6b3-068a-4b8d-9bdb-b07c15212c56 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:55:42.556763 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1242/2473] 10.222.0.81 () {58 vars in 1152 bytes} [Wed Jul 20 22:55:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:55:42.564451 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f4f6b3-068a-4b8d-9bdb-b07c15212c56 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:55:42.569495 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7fa659-6ff4-46ed-84dc-ff6e3030be80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:55:42.570253 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7fa659-6ff4-46ed-84dc-ff6e3030be80 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:55:42.570612 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7fa659-6ff4-46ed-84dc-ff6e3030be80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:55:42.570988 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7fa659-6ff4-46ed-84dc-ff6e3030be80 admin admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:55:42.571797 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7fa659-6ff4-46ed-84dc-ff6e3030be80 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:55:42.573819 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7fa659-6ff4-46ed-84dc-ff6e3030be80 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:55:42.674835 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1232/2474] 10.222.0.81 () {60 vars in 1152 bytes} [Wed Jul 20 22:55:42 2022] POST /identity/v3/projects => generated 381 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 22:55:42.678655 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca044f78-b6e8-4f40-88b6-986135d9987c admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:55:42.689271 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca044f78-b6e8-4f40-88b6-986135d9987c admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:55:42.754762 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bf5133-e3a6-4e8b-9b9f-44891df892f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:55:42.755481 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bf5133-e3a6-4e8b-9b9f-44891df892f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:55:42.755833 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bf5133-e3a6-4e8b-9b9f-44891df892f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:55:42.756322 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bf5133-e3a6-4e8b-9b9f-44891df892f8 admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:55:42.757290 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bf5133-e3a6-4e8b-9b9f-44891df892f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:55:42.760639 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bf5133-e3a6-4e8b-9b9f-44891df892f8 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:55:42.849157 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1243/2475] 10.222.0.81 () {60 vars in 1143 bytes} [Wed Jul 20 22:55:42 2022] POST /identity/v3/users => generated 291 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:55:42.853912 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7fa659-6ff4-46ed-84dc-ff6e3030be80 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:55:42.865363 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7fa659-6ff4-46ed-84dc-ff6e3030be80 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:55:42.870538 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c62fd62-0221-4257-93fa-1722aaaf86f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:55:42.871261 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c62fd62-0221-4257-93fa-1722aaaf86f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:55:42.871581 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c62fd62-0221-4257-93fa-1722aaaf86f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:55:42.871882 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c62fd62-0221-4257-93fa-1722aaaf86f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:55:42.873225 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c62fd62-0221-4257-93fa-1722aaaf86f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:55:42.874911 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c62fd62-0221-4257-93fa-1722aaaf86f7 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:55:42.887628 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1233/2476] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:55:42 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:55:42.892834 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bf5133-e3a6-4e8b-9b9f-44891df892f8 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:55:42.904002 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bf5133-e3a6-4e8b-9b9f-44891df892f8 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:55:42.908908 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566ef805-6289-4aa4-b854-4826ce309e49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:55:42.909593 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566ef805-6289-4aa4-b854-4826ce309e49 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:55:42.909883 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566ef805-6289-4aa4-b854-4826ce309e49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:55:42.910164 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566ef805-6289-4aa4-b854-4826ce309e49 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:55:42.910905 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566ef805-6289-4aa4-b854-4826ce309e49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:55:42.912586 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566ef805-6289-4aa4-b854-4826ce309e49 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:55:42.927057 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1244/2477] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:55:42 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:55:42.930507 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c62fd62-0221-4257-93fa-1722aaaf86f7 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:55:42.952641 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c62fd62-0221-4257-93fa-1722aaaf86f7 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:55:42.957439 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4078a0c3-19d5-490b-871a-4952579ef109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:55:42.958365 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4078a0c3-19d5-490b-871a-4952579ef109 admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:55:42.958628 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4078a0c3-19d5-490b-871a-4952579ef109 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:55:42.958904 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4078a0c3-19d5-490b-871a-4952579ef109 admin admin] PATH_INFO: `/v3/projects/37959f4f333c4675b761f128faad98af/users/9df90184169d4d78b40ca6f73017d8d0/roles/559942c4de1744169f8b5e0404ecde6b` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:55:42.977406 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4078a0c3-19d5-490b-871a-4952579ef109 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37959f4f333c4675b761f128faad98af, user_id=9df90184169d4d78b40ca6f73017d8d0, role_id=559942c4de1744169f8b5e0404ecde6b)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:55:42.980450 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4078a0c3-19d5-490b-871a-4952579ef109 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:55:43.375004 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1234/2478] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:55:42 2022] PUT /identity/v3/projects/37959f4f333c4675b761f128faad98af/users/9df90184169d4d78b40ca6f73017d8d0/roles/559942c4de1744169f8b5e0404ecde6b => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 22:55:43.385535 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761effe8-d86f-4d1d-b6c0-a85555454b13 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:55:43.385854 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761effe8-d86f-4d1d-b6c0-a85555454b13 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:55:43.386127 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761effe8-d86f-4d1d-b6c0-a85555454b13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:55:43.416991 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-761effe8-d86f-4d1d-b6c0-a85555454b13 None None] MFA Rules not processed for user `9df90184169d4d78b40ca6f73017d8d0`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:55:43.418302 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-761effe8-d86f-4d1d-b6c0-a85555454b13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:55:43.424180 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:55:43.496092 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1245/2479] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:55:43 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:55:43.500497 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4078a0c3-19d5-490b-871a-4952579ef109 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:55:43.573938 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4078a0c3-19d5-490b-871a-4952579ef109 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:55:43.578566 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7047b532-ec4e-41bd-9769-32965e65b884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:55:43.579248 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7047b532-ec4e-41bd-9769-32965e65b884 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:55:43.579534 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7047b532-ec4e-41bd-9769-32965e65b884 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:55:43.579810 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7047b532-ec4e-41bd-9769-32965e65b884 admin admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:55:43.580491 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7047b532-ec4e-41bd-9769-32965e65b884 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:55:43.582380 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7047b532-ec4e-41bd-9769-32965e65b884 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:55:43.707548 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1235/2480] 10.222.0.81 () {60 vars in 1152 bytes} [Wed Jul 20 22:55:43 2022] POST /identity/v3/projects => generated 379 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:55:43.711633 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761effe8-d86f-4d1d-b6c0-a85555454b13 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:55:43.725378 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761effe8-d86f-4d1d-b6c0-a85555454b13 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:55:43.796579 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc06ded9-415b-4028-8fad-82c6f749759a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:55:43.797294 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc06ded9-415b-4028-8fad-82c6f749759a admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:55:43.797675 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc06ded9-415b-4028-8fad-82c6f749759a admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:55:43.798138 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc06ded9-415b-4028-8fad-82c6f749759a admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:55:43.798932 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc06ded9-415b-4028-8fad-82c6f749759a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:55:43.801800 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc06ded9-415b-4028-8fad-82c6f749759a admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:55:43.898053 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1246/2481] 10.222.0.81 () {60 vars in 1143 bytes} [Wed Jul 20 22:55:43 2022] POST /identity/v3/users => generated 296 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:55:43.899514 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7047b532-ec4e-41bd-9769-32965e65b884 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:55:43.910807 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7047b532-ec4e-41bd-9769-32965e65b884 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:55:43.917637 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a6a50d-75bc-4d9d-85a0-5178afa00dfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:55:43.918498 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a6a50d-75bc-4d9d-85a0-5178afa00dfe admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:55:43.918901 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a6a50d-75bc-4d9d-85a0-5178afa00dfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:55:43.919361 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a6a50d-75bc-4d9d-85a0-5178afa00dfe admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:55:43.920728 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a6a50d-75bc-4d9d-85a0-5178afa00dfe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:55:43.922399 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a6a50d-75bc-4d9d-85a0-5178afa00dfe admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:55:43.937613 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1236/2482] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:55:43 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:55:43.938808 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc06ded9-415b-4028-8fad-82c6f749759a admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:55:43.951027 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc06ded9-415b-4028-8fad-82c6f749759a admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:55:43.955895 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d468e330-57ef-4c24-81cf-a0b7f026df53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:55:43.956808 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d468e330-57ef-4c24-81cf-a0b7f026df53 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:55:43.957100 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d468e330-57ef-4c24-81cf-a0b7f026df53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:55:43.957380 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d468e330-57ef-4c24-81cf-a0b7f026df53 admin admin] PATH_INFO: `/v3/projects/10f9e76f914646a2a20d5ea28fa5f5ad/users/3647b5ee6b0f48538056016fa0f738d7/roles/d89b6c92d35a401a8517f1d47f6be843` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:55:43.978630 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d468e330-57ef-4c24-81cf-a0b7f026df53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=10f9e76f914646a2a20d5ea28fa5f5ad, user_id=3647b5ee6b0f48538056016fa0f738d7, role_id=d89b6c92d35a401a8517f1d47f6be843)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:55:43.984942 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d468e330-57ef-4c24-81cf-a0b7f026df53 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:55:44.063887 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1247/2483] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:55:43 2022] PUT /identity/v3/projects/10f9e76f914646a2a20d5ea28fa5f5ad/users/3647b5ee6b0f48538056016fa0f738d7/roles/d89b6c92d35a401a8517f1d47f6be843 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:55:44.067955 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a6a50d-75bc-4d9d-85a0-5178afa00dfe admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:55:44.079049 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a6a50d-75bc-4d9d-85a0-5178afa00dfe admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:55:44.084272 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:55:44.140152 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543466d7-72b7-4deb-90bd-4d48b3a336e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:55:44.140835 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543466d7-72b7-4deb-90bd-4d48b3a336e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:55:44.141119 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543466d7-72b7-4deb-90bd-4d48b3a336e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:55:44.141399 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543466d7-72b7-4deb-90bd-4d48b3a336e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:55:44.142125 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543466d7-72b7-4deb-90bd-4d48b3a336e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:55:44.143837 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543466d7-72b7-4deb-90bd-4d48b3a336e0 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:55:44.155410 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1237/2484] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:55:44 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:55:44.159624 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d468e330-57ef-4c24-81cf-a0b7f026df53 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:55:44.170392 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d468e330-57ef-4c24-81cf-a0b7f026df53 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:55:44.175242 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dddf34d-054c-4bef-b496-89d9e92947ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:55:44.176261 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dddf34d-054c-4bef-b496-89d9e92947ae admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:55:44.176650 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dddf34d-054c-4bef-b496-89d9e92947ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:55:44.177050 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dddf34d-054c-4bef-b496-89d9e92947ae admin admin] PATH_INFO: `/v3/projects/10f9e76f914646a2a20d5ea28fa5f5ad/users/3647b5ee6b0f48538056016fa0f738d7/roles/d89b6c92d35a401a8517f1d47f6be843` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:55:44.179272 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dddf34d-054c-4bef-b496-89d9e92947ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=10f9e76f914646a2a20d5ea28fa5f5ad, user_id=3647b5ee6b0f48538056016fa0f738d7, role_id=d89b6c92d35a401a8517f1d47f6be843)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:55:44.181909 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dddf34d-054c-4bef-b496-89d9e92947ae admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:55:44.202249 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1248/2485] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:55:44 2022] PUT /identity/v3/projects/10f9e76f914646a2a20d5ea28fa5f5ad/users/3647b5ee6b0f48538056016fa0f738d7/roles/d89b6c92d35a401a8517f1d47f6be843 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:55:44.204465 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ef0668-86a4-48b7-b931-25f5ad26d191 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:55:44.204981 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ef0668-86a4-48b7-b931-25f5ad26d191 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:55:44.205347 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ef0668-86a4-48b7-b931-25f5ad26d191 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:55:44.235740 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-f4ef0668-86a4-48b7-b931-25f5ad26d191 None None] MFA Rules not processed for user `3647b5ee6b0f48538056016fa0f738d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:55:44.236993 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-f4ef0668-86a4-48b7-b931-25f5ad26d191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:55:44.310600 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1238/2486] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:55:44 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 22:55:44.323649 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dddf34d-054c-4bef-b496-89d9e92947ae admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:55:44.325413 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-3dddf34d-054c-4bef-b496-89d9e92947ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:55:44.404819 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:55:44.425085 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dddf34d-054c-4bef-b496-89d9e92947ae admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:55:44.428894 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af12fea1-651b-4084-b041-ba42d8c1963e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecf4db289fba4e4a806198c62d31f7c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:55:44.429556 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af12fea1-651b-4084-b041-ba42d8c1963e service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:55:44.429843 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af12fea1-651b-4084-b041-ba42d8c1963e service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:55:44.434709 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af12fea1-651b-4084-b041-ba42d8c1963e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:55:44.444571 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af12fea1-651b-4084-b041-ba42d8c1963e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:55:44.446003 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af12fea1-651b-4084-b041-ba42d8c1963e service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:55:44.456705 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1249/2487] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:55:44 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:55:44.684043 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ef0668-86a4-48b7-b931-25f5ad26d191 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:55:44.695165 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ef0668-86a4-48b7-b931-25f5ad26d191 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:55:44.698883 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d13facb-181c-4d9f-bd99-6d59167c24a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecf4db289fba4e4a806198c62d31f7c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:55:44.699575 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d13facb-181c-4d9f-bd99-6d59167c24a9 service cinder] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:55:44.702378 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d13facb-181c-4d9f-bd99-6d59167c24a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:55:44.702378 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d13facb-181c-4d9f-bd99-6d59167c24a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:55:44.757772 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d13facb-181c-4d9f-bd99-6d59167c24a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:55:44.759488 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d13facb-181c-4d9f-bd99-6d59167c24a9 service cinder] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:55:44.768384 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1239/2488] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:55:44 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:55:50.867168 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e44d2e-882e-4044-a6e5-3d4ee4b8921d None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:55:50.867671 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e44d2e-882e-4044-a6e5-3d4ee4b8921d None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:55:50.867770 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e44d2e-882e-4044-a6e5-3d4ee4b8921d None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:55:50.869115 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1250/2489] 10.222.0.81 () {54 vars in 921 bytes} [Wed Jul 20 22:55:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:55:50.875424 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9f3b9a-c5fc-40a6-97d3-4f3c6e000a55 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:55:50.875864 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9f3b9a-c5fc-40a6-97d3-4f3c6e000a55 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:55:50.876153 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9f3b9a-c5fc-40a6-97d3-4f3c6e000a55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:55:50.915349 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-fd9f3b9a-c5fc-40a6-97d3-4f3c6e000a55 None None] MFA Rules not processed for user `7829324db2574d38b18c749d0559177f`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:55:50.916457 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-fd9f3b9a-c5fc-40a6-97d3-4f3c6e000a55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:55:50.926681 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:55:50.983421 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1240/2490] 10.222.0.81 () {58 vars in 1018 bytes} [Wed Jul 20 22:55:50 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 22:55:50.994702 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e44d2e-882e-4044-a6e5-3d4ee4b8921d None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:55:50.996540 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-12e44d2e-882e-4044-a6e5-3d4ee4b8921d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:55:51.054366 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e44d2e-882e-4044-a6e5-3d4ee4b8921d None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:55:51.058081 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b68919d-0a83-47a4-8e15-366601998c06 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c44640ea0014426a9523bb1c457778b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:55:51.058734 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b68919d-0a83-47a4-8e15-366601998c06 service glance] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:55:51.059070 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b68919d-0a83-47a4-8e15-366601998c06 service glance] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:55:51.059353 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b68919d-0a83-47a4-8e15-366601998c06 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:55:51.069754 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b68919d-0a83-47a4-8e15-366601998c06 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:55:51.071199 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b68919d-0a83-47a4-8e15-366601998c06 service glance] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:55:51.082788 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1251/2491] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:55:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:55:51.319292 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa703faa-eff4-43da-93ee-caf52a0e7c58 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:55:51.319585 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa703faa-eff4-43da-93ee-caf52a0e7c58 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:55:51.319870 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa703faa-eff4-43da-93ee-caf52a0e7c58 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:55:51.321239 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1241/2492] 10.222.0.81 () {54 vars in 921 bytes} [Wed Jul 20 22:55:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:55:51.327152 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bb39f9-120d-4572-821a-e39fc8abe2fb None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:55:51.327488 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bb39f9-120d-4572-821a-e39fc8abe2fb None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:55:51.327776 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bb39f9-120d-4572-821a-e39fc8abe2fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:55:51.364690 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-78bb39f9-120d-4572-821a-e39fc8abe2fb None None] MFA Rules not processed for user `333c2b1b11cc4c478abfd965a72e9d66`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:55:51.365932 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-78bb39f9-120d-4572-821a-e39fc8abe2fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:55:51.423410 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1252/2493] 10.222.0.81 () {58 vars in 1018 bytes} [Wed Jul 20 22:55:51 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:55:51.457085 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa703faa-eff4-43da-93ee-caf52a0e7c58 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:55:51.460657 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-aa703faa-eff4-43da-93ee-caf52a0e7c58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:55:51.475760 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa703faa-eff4-43da-93ee-caf52a0e7c58 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:55:51.479952 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1904a72-e8b0-4907-b386-3fd0be0323c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:55:51.480678 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1904a72-e8b0-4907-b386-3fd0be0323c9 service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:55:51.481037 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1904a72-e8b0-4907-b386-3fd0be0323c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:55:51.481409 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1904a72-e8b0-4907-b386-3fd0be0323c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:55:51.489263 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1904a72-e8b0-4907-b386-3fd0be0323c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:55:51.490688 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1904a72-e8b0-4907-b386-3fd0be0323c9 service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:55:51.499291 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1242/2494] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:55:51 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:55:51.604395 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab0a372-35ba-4709-9a0b-cbaaf0c9c775 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:55:51.604692 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab0a372-35ba-4709-9a0b-cbaaf0c9c775 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:55:51.604979 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab0a372-35ba-4709-9a0b-cbaaf0c9c775 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:55:51.606391 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1253/2495] 10.222.0.81 () {54 vars in 921 bytes} [Wed Jul 20 22:55:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:55:51.611785 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e7d1fb-ad06-4ebb-99de-70059c895ef3 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:55:51.612074 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e7d1fb-ad06-4ebb-99de-70059c895ef3 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:55:51.612354 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e7d1fb-ad06-4ebb-99de-70059c895ef3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:55:51.631792 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-09e7d1fb-ad06-4ebb-99de-70059c895ef3 None None] MFA Rules not processed for user `7829324db2574d38b18c749d0559177f`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:55:51.632926 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-09e7d1fb-ad06-4ebb-99de-70059c895ef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:55:51.643480 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1243/2496] 10.222.0.81 () {58 vars in 1018 bytes} [Wed Jul 20 22:55:51 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:55:51.653725 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab0a372-35ba-4709-9a0b-cbaaf0c9c775 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:55:51.664474 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab0a372-35ba-4709-9a0b-cbaaf0c9c775 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:55:51.667953 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc2fbcc-947f-4e32-97a9-8e0ae00853fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:55:51.668613 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc2fbcc-947f-4e32-97a9-8e0ae00853fa service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:55:51.668921 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc2fbcc-947f-4e32-97a9-8e0ae00853fa service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:55:51.669196 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc2fbcc-947f-4e32-97a9-8e0ae00853fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:55:51.679677 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc2fbcc-947f-4e32-97a9-8e0ae00853fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:55:51.681103 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc2fbcc-947f-4e32-97a9-8e0ae00853fa service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:55:51.692142 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1254/2497] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:55:51 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:55:55.952212 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc229254-7362-4824-aeaf-17af70ecaf7a None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:55:55.952698 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc229254-7362-4824-aeaf-17af70ecaf7a None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:55:55.952803 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc229254-7362-4824-aeaf-17af70ecaf7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:55:56.001177 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-bc229254-7362-4824-aeaf-17af70ecaf7a None None] MFA Rules not processed for user `90915e1264324f16bc7bae7e46e0c97b`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:55:56.002417 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-bc229254-7362-4824-aeaf-17af70ecaf7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:55:56.067195 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1244/2498] 10.222.0.81 () {58 vars in 1009 bytes} [Wed Jul 20 22:55:55 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:55:56.079789 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc2fbcc-947f-4e32-97a9-8e0ae00853fa service neutron] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:55:56.081286 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-ebc2fbcc-947f-4e32-97a9-8e0ae00853fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:55:56.166684 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:55:56.207080 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc2fbcc-947f-4e32-97a9-8e0ae00853fa service neutron] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:55:56.212717 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d31ba8-d8e0-458d-b4f1-66e64c460bb4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70e97436a08543a180225561ef8bc817', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:55:56.213392 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d31ba8-d8e0-458d-b4f1-66e64c460bb4 service swift] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:55:56.213684 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d31ba8-d8e0-458d-b4f1-66e64c460bb4 service swift] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:55:56.213997 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d31ba8-d8e0-458d-b4f1-66e64c460bb4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:55:56.224038 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d31ba8-d8e0-458d-b4f1-66e64c460bb4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:55:56.225778 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d31ba8-d8e0-458d-b4f1-66e64c460bb4 service swift] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:55:56.235527 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1255/2499] 10.222.0.81 () {60 vars in 1384 bytes} [Wed Jul 20 22:55:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 22:55:57.637305 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc229254-7362-4824-aeaf-17af70ecaf7a None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:55:57.701827 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc229254-7362-4824-aeaf-17af70ecaf7a None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:55:57.706694 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4ef2bd-e25c-40a5-b58b-c258a16d6a30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:55:57.712032 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4ef2bd-e25c-40a5-b58b-c258a16d6a30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:55:57.712325 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4ef2bd-e25c-40a5-b58b-c258a16d6a30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:55:57.712613 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4ef2bd-e25c-40a5-b58b-c258a16d6a30 admin admin] PATH_INFO: `/v3/users/9df90184169d4d78b40ca6f73017d8d0` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:55:57.714006 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4ef2bd-e25c-40a5-b58b-c258a16d6a30 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9df90184169d4d78b40ca6f73017d8d0)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:55:57.716393 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4ef2bd-e25c-40a5-b58b-c258a16d6a30 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:55:57.718406 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:55:57.895534 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-6b4ef2bd-e25c-40a5-b58b-c258a16d6a30 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9df90184169d4d78b40ca6f73017d8d0'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:55:57.896214 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-6b4ef2bd-e25c-40a5-b58b-c258a16d6a30 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9df90184169d4d78b40ca6f73017d8d0'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:55:57.896813 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-6b4ef2bd-e25c-40a5-b58b-c258a16d6a30 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9df90184169d4d78b40ca6f73017d8d0'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:55:57.905052 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-6b4ef2bd-e25c-40a5-b58b-c258a16d6a30 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9df90184169d4d78b40ca6f73017d8d0'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:55:57.979682 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-6b4ef2bd-e25c-40a5-b58b-c258a16d6a30 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9df90184169d4d78b40ca6f73017d8d0'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:55:58.004986 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1245/2500] 10.222.0.81 () {58 vars in 1223 bytes} [Wed Jul 20 22:55:57 2022] DELETE /identity/v3/users/9df90184169d4d78b40ca6f73017d8d0 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:55:58.010064 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12940f1c-0e8f-4209-8a9a-fecfd1f4cda5 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:55:58.010601 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12940f1c-0e8f-4209-8a9a-fecfd1f4cda5 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:55:58.010687 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12940f1c-0e8f-4209-8a9a-fecfd1f4cda5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:55:58.036057 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-12940f1c-0e8f-4209-8a9a-fecfd1f4cda5 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:55:58.037323 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-12940f1c-0e8f-4209-8a9a-fecfd1f4cda5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:55:58.114869 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1256/2501] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:55:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:55:58.125082 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4ef2bd-e25c-40a5-b58b-c258a16d6a30 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:55:58.126774 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-6b4ef2bd-e25c-40a5-b58b-c258a16d6a30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:55:58.191940 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4ef2bd-e25c-40a5-b58b-c258a16d6a30 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:55:58.196727 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2e9ae2-9531-4464-b7e1-00176edae9bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:55:58.196980 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2e9ae2-9531-4464-b7e1-00176edae9bf service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:55:58.197202 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2e9ae2-9531-4464-b7e1-00176edae9bf service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:55:58.197485 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2e9ae2-9531-4464-b7e1-00176edae9bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:55:58.207733 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2e9ae2-9531-4464-b7e1-00176edae9bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:55:58.209583 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2e9ae2-9531-4464-b7e1-00176edae9bf service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:55:58.221663 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1246/2502] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:55:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:55:58.649948 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12940f1c-0e8f-4209-8a9a-fecfd1f4cda5 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:55:58.652808 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-12940f1c-0e8f-4209-8a9a-fecfd1f4cda5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:55:58.664817 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12940f1c-0e8f-4209-8a9a-fecfd1f4cda5 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:55:58.670013 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11768538-ad45-44ac-834f-76384c870416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:55:58.671201 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11768538-ad45-44ac-834f-76384c870416 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:55:58.671692 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11768538-ad45-44ac-834f-76384c870416 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:55:58.672185 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11768538-ad45-44ac-834f-76384c870416 admin admin] PATH_INFO: `/v3/projects/37959f4f333c4675b761f128faad98af` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:55:58.673571 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11768538-ad45-44ac-834f-76384c870416 admin admin] RBAC: Authorizing `identity:delete_project(project_id=37959f4f333c4675b761f128faad98af)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:55:58.675924 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11768538-ad45-44ac-834f-76384c870416 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:55:58.783552 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-11768538-ad45-44ac-834f-76384c870416 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37959f4f333c4675b761f128faad98af', 'request_id': 'req-11768538-ad45-44ac-834f-76384c870416'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:55:58.791066 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-11768538-ad45-44ac-834f-76384c870416 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37959f4f333c4675b761f128faad98af', 'request_id': 'req-11768538-ad45-44ac-834f-76384c870416'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:55:58.871400 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-11768538-ad45-44ac-834f-76384c870416 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37959f4f333c4675b761f128faad98af', 'request_id': 'req-11768538-ad45-44ac-834f-76384c870416'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:55:58.887598 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-11768538-ad45-44ac-834f-76384c870416 admin admin] The token cache is being invalidate because project 37959f4f333c4675b761f128faad98af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:55:58.887938 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-11768538-ad45-44ac-834f-76384c870416 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:55:58.888716 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-11768538-ad45-44ac-834f-76384c870416 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:55:58.890517 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1257/2503] 10.222.0.81 () {58 vars in 1232 bytes} [Wed Jul 20 22:55:58 2022] DELETE /identity/v3/projects/37959f4f333c4675b761f128faad98af => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 22:55:58.894373 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2e9ae2-9531-4464-b7e1-00176edae9bf service neutron] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:55:58.905246 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2e9ae2-9531-4464-b7e1-00176edae9bf service neutron] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:55:58.972149 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b7128d-e24b-4bfd-a7c1-7c3311f8cb17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:55:58.973234 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b7128d-e24b-4bfd-a7c1-7c3311f8cb17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:55:58.977798 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b7128d-e24b-4bfd-a7c1-7c3311f8cb17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:55:58.977798 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b7128d-e24b-4bfd-a7c1-7c3311f8cb17 admin admin] PATH_INFO: `/v3/users/3647b5ee6b0f48538056016fa0f738d7` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:55:58.977798 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b7128d-e24b-4bfd-a7c1-7c3311f8cb17 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3647b5ee6b0f48538056016fa0f738d7)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:55:58.977798 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b7128d-e24b-4bfd-a7c1-7c3311f8cb17 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:55:59.000340 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:55:59.196847 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-63b7128d-e24b-4bfd-a7c1-7c3311f8cb17 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3647b5ee6b0f48538056016fa0f738d7'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:55:59.197236 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-63b7128d-e24b-4bfd-a7c1-7c3311f8cb17 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3647b5ee6b0f48538056016fa0f738d7'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:55:59.198040 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-63b7128d-e24b-4bfd-a7c1-7c3311f8cb17 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3647b5ee6b0f48538056016fa0f738d7'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:55:59.212078 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-63b7128d-e24b-4bfd-a7c1-7c3311f8cb17 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3647b5ee6b0f48538056016fa0f738d7'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:55:59.271398 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-63b7128d-e24b-4bfd-a7c1-7c3311f8cb17 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3647b5ee6b0f48538056016fa0f738d7'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:55:59.300237 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1247/2504] 10.222.0.81 () {58 vars in 1223 bytes} [Wed Jul 20 22:55:58 2022] DELETE /identity/v3/users/3647b5ee6b0f48538056016fa0f738d7 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:55:59.713296 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11768538-ad45-44ac-834f-76384c870416 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:55:59.714988 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-11768538-ad45-44ac-834f-76384c870416 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:55:59.730099 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11768538-ad45-44ac-834f-76384c870416 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:55:59.843726 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8510b72f-e509-4263-a1ee-6e6282bb0f4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:55:59.844723 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8510b72f-e509-4263-a1ee-6e6282bb0f4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:55:59.845074 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8510b72f-e509-4263-a1ee-6e6282bb0f4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:55:59.845357 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8510b72f-e509-4263-a1ee-6e6282bb0f4c admin admin] PATH_INFO: `/v3/projects/10f9e76f914646a2a20d5ea28fa5f5ad` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:55:59.846685 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8510b72f-e509-4263-a1ee-6e6282bb0f4c admin admin] RBAC: Authorizing `identity:delete_project(project_id=10f9e76f914646a2a20d5ea28fa5f5ad)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:55:59.849051 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8510b72f-e509-4263-a1ee-6e6282bb0f4c admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:55:59.853000 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:56:00.015764 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-8510b72f-e509-4263-a1ee-6e6282bb0f4c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10f9e76f914646a2a20d5ea28fa5f5ad', 'request_id': 'req-8510b72f-e509-4263-a1ee-6e6282bb0f4c'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:56:00.022346 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-8510b72f-e509-4263-a1ee-6e6282bb0f4c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10f9e76f914646a2a20d5ea28fa5f5ad', 'request_id': 'req-8510b72f-e509-4263-a1ee-6e6282bb0f4c'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:56:00.095240 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-8510b72f-e509-4263-a1ee-6e6282bb0f4c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10f9e76f914646a2a20d5ea28fa5f5ad', 'request_id': 'req-8510b72f-e509-4263-a1ee-6e6282bb0f4c'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:56:00.110981 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-8510b72f-e509-4263-a1ee-6e6282bb0f4c admin admin] The token cache is being invalidate because project 10f9e76f914646a2a20d5ea28fa5f5ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:56:00.111861 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-8510b72f-e509-4263-a1ee-6e6282bb0f4c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:56:00.113375 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-8510b72f-e509-4263-a1ee-6e6282bb0f4c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:56:00.118862 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1258/2505] 10.222.0.81 () {58 vars in 1232 bytes} [Wed Jul 20 22:55:59 2022] DELETE /identity/v3/projects/10f9e76f914646a2a20d5ea28fa5f5ad => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:56:00.120544 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f99208a-9d58-4465-8e0a-7d074aee8f4e None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:56:00.121081 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f99208a-9d58-4465-8e0a-7d074aee8f4e None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:56:00.121207 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f99208a-9d58-4465-8e0a-7d074aee8f4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:56:00.144660 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-5f99208a-9d58-4465-8e0a-7d074aee8f4e None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:56:00.146044 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-5f99208a-9d58-4465-8e0a-7d074aee8f4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:56:00.219974 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1248/2506] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:56:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:56:00.228520 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0962c2da-ac5a-47ce-b11a-4322498a2d6e None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:56:00.228832 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0962c2da-ac5a-47ce-b11a-4322498a2d6e None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:56:00.229121 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0962c2da-ac5a-47ce-b11a-4322498a2d6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:56:00.256019 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-0962c2da-ac5a-47ce-b11a-4322498a2d6e None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:56:00.257521 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-0962c2da-ac5a-47ce-b11a-4322498a2d6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:56:00.269402 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1259/2507] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:56:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:56:00.273785 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f99208a-9d58-4465-8e0a-7d074aee8f4e None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:56:00.286019 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f99208a-9d58-4465-8e0a-7d074aee8f4e None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:56:00.290785 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42d44e4-70c4-4002-8f39-a5af57eb95ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:56:00.291504 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42d44e4-70c4-4002-8f39-a5af57eb95ae admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:56:00.291793 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42d44e4-70c4-4002-8f39-a5af57eb95ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:56:00.292075 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42d44e4-70c4-4002-8f39-a5af57eb95ae admin admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:56:00.292809 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42d44e4-70c4-4002-8f39-a5af57eb95ae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:56:00.293152 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42d44e4-70c4-4002-8f39-a5af57eb95ae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:56:00.294786 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42d44e4-70c4-4002-8f39-a5af57eb95ae admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:56:00.308356 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1249/2508] 10.222.0.81 () {58 vars in 1152 bytes} [Wed Jul 20 22:56:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:56:00.312201 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0962c2da-ac5a-47ce-b11a-4322498a2d6e None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:56:00.322275 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0962c2da-ac5a-47ce-b11a-4322498a2d6e None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:56:00.334257 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c816a71-bc20-4555-b3f5-f1a17cae46d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:56:00.335099 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c816a71-bc20-4555-b3f5-f1a17cae46d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:56:00.335494 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c816a71-bc20-4555-b3f5-f1a17cae46d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:56:00.335897 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c816a71-bc20-4555-b3f5-f1a17cae46d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:56:00.336882 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c816a71-bc20-4555-b3f5-f1a17cae46d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:56:00.338900 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c816a71-bc20-4555-b3f5-f1a17cae46d1 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:56:00.447875 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1260/2509] 10.222.0.81 () {60 vars in 1152 bytes} [Wed Jul 20 22:56:00 2022] POST /identity/v3/projects => generated 363 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:56:00.452640 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42d44e4-70c4-4002-8f39-a5af57eb95ae admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:56:00.464341 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42d44e4-70c4-4002-8f39-a5af57eb95ae admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:56:00.525232 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04aa7901-65e6-431c-a197-77f85a476341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:56:00.525913 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aa7901-65e6-431c-a197-77f85a476341 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:56:00.526196 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aa7901-65e6-431c-a197-77f85a476341 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:56:00.526475 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aa7901-65e6-431c-a197-77f85a476341 admin admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:56:00.527201 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04aa7901-65e6-431c-a197-77f85a476341 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:56:00.529182 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04aa7901-65e6-431c-a197-77f85a476341 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:56:00.640087 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1250/2510] 10.222.0.81 () {60 vars in 1143 bytes} [Wed Jul 20 22:56:00 2022] POST /identity/v3/users => generated 282 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:56:00.644244 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c816a71-bc20-4555-b3f5-f1a17cae46d1 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:56:00.658145 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c816a71-bc20-4555-b3f5-f1a17cae46d1 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:56:00.665150 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903785af-9389-4597-82f7-7f4634331386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:56:00.665942 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903785af-9389-4597-82f7-7f4634331386 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:56:00.666257 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903785af-9389-4597-82f7-7f4634331386 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:56:00.666546 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903785af-9389-4597-82f7-7f4634331386 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:56:00.667422 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903785af-9389-4597-82f7-7f4634331386 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:56:00.669048 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903785af-9389-4597-82f7-7f4634331386 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:56:00.687299 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1261/2511] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:56:00 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:56:00.692746 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04aa7901-65e6-431c-a197-77f85a476341 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:56:00.705753 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04aa7901-65e6-431c-a197-77f85a476341 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:56:00.711904 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0e7155-2664-45cb-a975-6da5a6da915b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:56:00.712776 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0e7155-2664-45cb-a975-6da5a6da915b admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:56:00.713125 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0e7155-2664-45cb-a975-6da5a6da915b admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:56:00.713647 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0e7155-2664-45cb-a975-6da5a6da915b admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:56:00.714459 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0e7155-2664-45cb-a975-6da5a6da915b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:56:00.716177 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0e7155-2664-45cb-a975-6da5a6da915b admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:56:00.728864 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1251/2512] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:56:00 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 22:56:00.733910 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903785af-9389-4597-82f7-7f4634331386 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:56:00.744740 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903785af-9389-4597-82f7-7f4634331386 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:56:00.750933 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12bff21-2ffd-413a-928e-cf30771f6385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:56:00.751941 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12bff21-2ffd-413a-928e-cf30771f6385 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:56:00.752231 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12bff21-2ffd-413a-928e-cf30771f6385 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:56:00.752601 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12bff21-2ffd-413a-928e-cf30771f6385 admin admin] PATH_INFO: `/v3/projects/0fac226216fc494fb6452d619e5db30d/users/79182f3372e24185847b1ea62dd14aa7/roles/559942c4de1744169f8b5e0404ecde6b` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:56:00.769653 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12bff21-2ffd-413a-928e-cf30771f6385 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0fac226216fc494fb6452d619e5db30d, user_id=79182f3372e24185847b1ea62dd14aa7, role_id=559942c4de1744169f8b5e0404ecde6b)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:56:00.772410 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12bff21-2ffd-413a-928e-cf30771f6385 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:56:01.307169 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1262/2513] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:56:00 2022] PUT /identity/v3/projects/0fac226216fc494fb6452d619e5db30d/users/79182f3372e24185847b1ea62dd14aa7/roles/559942c4de1744169f8b5e0404ecde6b => generated 0 bytes in 574 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:56:01.317212 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f76170d-d7d7-41da-a894-c74a67608d40 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:56:01.317877 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f76170d-d7d7-41da-a894-c74a67608d40 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:56:01.317877 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f76170d-d7d7-41da-a894-c74a67608d40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:56:01.330397 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:56:01.345804 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-4f76170d-d7d7-41da-a894-c74a67608d40 None None] MFA Rules not processed for user `79182f3372e24185847b1ea62dd14aa7`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:56:01.369107 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-4f76170d-d7d7-41da-a894-c74a67608d40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:56:01.426389 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1252/2514] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:56:01 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:56:01.427486 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12bff21-2ffd-413a-928e-cf30771f6385 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:56:01.506774 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12bff21-2ffd-413a-928e-cf30771f6385 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:56:01.512790 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8700a7ee-afe3-4c2b-889f-7853ffbca9dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:56:01.514246 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8700a7ee-afe3-4c2b-889f-7853ffbca9dd admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:56:01.514792 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8700a7ee-afe3-4c2b-889f-7853ffbca9dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:56:01.514792 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8700a7ee-afe3-4c2b-889f-7853ffbca9dd admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:56:01.515626 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8700a7ee-afe3-4c2b-889f-7853ffbca9dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:56:01.517763 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8700a7ee-afe3-4c2b-889f-7853ffbca9dd admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:56:01.642932 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1263/2515] 10.222.0.81 () {60 vars in 1152 bytes} [Wed Jul 20 22:56:01 2022] POST /identity/v3/projects => generated 365 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 22:56:01.644174 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f76170d-d7d7-41da-a894-c74a67608d40 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:56:01.655997 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f76170d-d7d7-41da-a894-c74a67608d40 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:56:01.734112 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee630b8-460e-4ecd-ad88-c6480e5fc868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:56:01.734800 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee630b8-460e-4ecd-ad88-c6480e5fc868 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:56:01.735118 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee630b8-460e-4ecd-ad88-c6480e5fc868 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:56:01.735401 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee630b8-460e-4ecd-ad88-c6480e5fc868 admin admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:56:01.736127 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee630b8-460e-4ecd-ad88-c6480e5fc868 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:56:01.738092 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee630b8-460e-4ecd-ad88-c6480e5fc868 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:56:01.837531 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1253/2516] 10.222.0.81 () {60 vars in 1143 bytes} [Wed Jul 20 22:56:01 2022] POST /identity/v3/users => generated 289 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:56:01.841521 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8700a7ee-afe3-4c2b-889f-7853ffbca9dd admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:56:01.852572 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8700a7ee-afe3-4c2b-889f-7853ffbca9dd admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:56:01.857581 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4830c1-39af-47a0-a9e8-7de8ca8d66f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:56:01.858331 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4830c1-39af-47a0-a9e8-7de8ca8d66f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:56:01.858619 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4830c1-39af-47a0-a9e8-7de8ca8d66f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:56:01.858903 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4830c1-39af-47a0-a9e8-7de8ca8d66f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:56:01.859815 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4830c1-39af-47a0-a9e8-7de8ca8d66f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:56:01.861434 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4830c1-39af-47a0-a9e8-7de8ca8d66f8 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:56:01.874621 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1264/2517] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:56:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:56:01.878444 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee630b8-460e-4ecd-ad88-c6480e5fc868 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:56:01.889990 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee630b8-460e-4ecd-ad88-c6480e5fc868 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:56:01.894927 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be530170-ddfd-47c9-9593-e3d970d244bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:56:01.897173 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be530170-ddfd-47c9-9593-e3d970d244bc admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:56:01.897173 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be530170-ddfd-47c9-9593-e3d970d244bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:56:01.897173 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be530170-ddfd-47c9-9593-e3d970d244bc admin admin] PATH_INFO: `/v3/projects/48504ae2828047a1baf5dfde5be82d00/users/b031a699050b4bfebe8170fe39874673/roles/d89b6c92d35a401a8517f1d47f6be843` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:56:01.913393 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be530170-ddfd-47c9-9593-e3d970d244bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=48504ae2828047a1baf5dfde5be82d00, user_id=b031a699050b4bfebe8170fe39874673, role_id=d89b6c92d35a401a8517f1d47f6be843)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:56:01.916311 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be530170-ddfd-47c9-9593-e3d970d244bc admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:56:02.004250 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1254/2518] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:56:01 2022] PUT /identity/v3/projects/48504ae2828047a1baf5dfde5be82d00/users/b031a699050b4bfebe8170fe39874673/roles/d89b6c92d35a401a8517f1d47f6be843 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:56:02.008912 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4830c1-39af-47a0-a9e8-7de8ca8d66f8 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:56:02.015479 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:56:02.021085 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4830c1-39af-47a0-a9e8-7de8ca8d66f8 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:56:02.089428 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51db3c44-3d0c-421e-b1db-7dd8caf28d0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:56:02.090589 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51db3c44-3d0c-421e-b1db-7dd8caf28d0e admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:56:02.091158 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51db3c44-3d0c-421e-b1db-7dd8caf28d0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:56:02.091712 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51db3c44-3d0c-421e-b1db-7dd8caf28d0e admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:56:02.092672 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51db3c44-3d0c-421e-b1db-7dd8caf28d0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:56:02.094799 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51db3c44-3d0c-421e-b1db-7dd8caf28d0e admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:56:02.108288 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1265/2519] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:56:02 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:56:02.113464 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be530170-ddfd-47c9-9593-e3d970d244bc admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:56:02.126969 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be530170-ddfd-47c9-9593-e3d970d244bc admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:56:02.132901 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37084ac-b7ca-43da-9f26-2b0c26192211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:56:02.134040 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37084ac-b7ca-43da-9f26-2b0c26192211 admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:56:02.134648 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37084ac-b7ca-43da-9f26-2b0c26192211 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:56:02.134991 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37084ac-b7ca-43da-9f26-2b0c26192211 admin admin] PATH_INFO: `/v3/projects/48504ae2828047a1baf5dfde5be82d00/users/b031a699050b4bfebe8170fe39874673/roles/d89b6c92d35a401a8517f1d47f6be843` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:56:02.137272 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37084ac-b7ca-43da-9f26-2b0c26192211 admin admin] RBAC: Authorizing `identity:create_grant(project_id=48504ae2828047a1baf5dfde5be82d00, user_id=b031a699050b4bfebe8170fe39874673, role_id=d89b6c92d35a401a8517f1d47f6be843)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:56:02.146264 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37084ac-b7ca-43da-9f26-2b0c26192211 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:56:02.159063 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1255/2520] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:56:02 2022] PUT /identity/v3/projects/48504ae2828047a1baf5dfde5be82d00/users/b031a699050b4bfebe8170fe39874673/roles/d89b6c92d35a401a8517f1d47f6be843 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:56:02.176205 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8ea552-df49-4b30-aca9-d3b96dbb26eb None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:56:02.176589 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8ea552-df49-4b30-aca9-d3b96dbb26eb None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:56:02.176940 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8ea552-df49-4b30-aca9-d3b96dbb26eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:56:02.223394 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-ba8ea552-df49-4b30-aca9-d3b96dbb26eb None None] MFA Rules not processed for user `b031a699050b4bfebe8170fe39874673`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:56:02.224782 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-ba8ea552-df49-4b30-aca9-d3b96dbb26eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:56:02.302719 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1266/2521] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:56:02 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:56:02.315076 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37084ac-b7ca-43da-9f26-2b0c26192211 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:56:02.316914 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-e37084ac-b7ca-43da-9f26-2b0c26192211 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:56:02.377517 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:56:02.381955 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37084ac-b7ca-43da-9f26-2b0c26192211 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:56:02.386281 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee91d8d-2db0-4f49-ab50-f909bccdd07d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecf4db289fba4e4a806198c62d31f7c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:56:02.387005 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee91d8d-2db0-4f49-ab50-f909bccdd07d service cinder] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:56:02.387336 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee91d8d-2db0-4f49-ab50-f909bccdd07d service cinder] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:56:02.387573 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee91d8d-2db0-4f49-ab50-f909bccdd07d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:56:02.399603 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee91d8d-2db0-4f49-ab50-f909bccdd07d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:56:02.401106 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee91d8d-2db0-4f49-ab50-f909bccdd07d service cinder] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:56:02.411320 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1256/2522] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:56:02 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:56:02.640567 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8ea552-df49-4b30-aca9-d3b96dbb26eb None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:56:02.653409 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8ea552-df49-4b30-aca9-d3b96dbb26eb None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:56:02.663108 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5683ab3-2896-42d0-abab-123a0b9eee49 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecf4db289fba4e4a806198c62d31f7c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:56:02.664140 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5683ab3-2896-42d0-abab-123a0b9eee49 service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:56:02.664435 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5683ab3-2896-42d0-abab-123a0b9eee49 service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:56:02.664775 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5683ab3-2896-42d0-abab-123a0b9eee49 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:56:02.758269 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5683ab3-2896-42d0-abab-123a0b9eee49 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:56:02.761509 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5683ab3-2896-42d0-abab-123a0b9eee49 service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:56:02.773008 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1267/2523] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:56:02 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:56:06.176980 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee91d8d-2db0-4f49-ab50-f909bccdd07d service cinder] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:56:06.248085 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee91d8d-2db0-4f49-ab50-f909bccdd07d service cinder] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:56:06.253543 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9282b12e-92ef-459a-8318-b10e4368c6da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:56:06.255661 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9282b12e-92ef-459a-8318-b10e4368c6da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:56:06.255945 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9282b12e-92ef-459a-8318-b10e4368c6da admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:56:06.256222 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9282b12e-92ef-459a-8318-b10e4368c6da admin admin] PATH_INFO: `/v3/users/79182f3372e24185847b1ea62dd14aa7` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:56:06.259245 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9282b12e-92ef-459a-8318-b10e4368c6da admin admin] RBAC: Authorizing `identity:delete_user(user_id=79182f3372e24185847b1ea62dd14aa7)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:56:06.261452 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9282b12e-92ef-459a-8318-b10e4368c6da admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:56:06.416690 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-9282b12e-92ef-459a-8318-b10e4368c6da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79182f3372e24185847b1ea62dd14aa7'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:56:06.417340 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-9282b12e-92ef-459a-8318-b10e4368c6da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79182f3372e24185847b1ea62dd14aa7'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:56:06.418244 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-9282b12e-92ef-459a-8318-b10e4368c6da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79182f3372e24185847b1ea62dd14aa7'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:56:06.426609 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-9282b12e-92ef-459a-8318-b10e4368c6da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79182f3372e24185847b1ea62dd14aa7'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:56:06.482269 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-9282b12e-92ef-459a-8318-b10e4368c6da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79182f3372e24185847b1ea62dd14aa7'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:56:06.500542 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1257/2524] 10.222.0.81 () {58 vars in 1223 bytes} [Wed Jul 20 22:56:06 2022] DELETE /identity/v3/users/79182f3372e24185847b1ea62dd14aa7 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:56:06.505134 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab103ead-3a37-42c0-859e-3b121d497d34 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:56:06.505636 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab103ead-3a37-42c0-859e-3b121d497d34 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:56:06.505741 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab103ead-3a37-42c0-859e-3b121d497d34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:56:06.523148 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-ab103ead-3a37-42c0-859e-3b121d497d34 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:56:06.524379 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-ab103ead-3a37-42c0-859e-3b121d497d34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:56:06.540857 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:56:06.581993 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1268/2525] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:56:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:56:06.591730 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9282b12e-92ef-459a-8318-b10e4368c6da admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:56:06.593821 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-9282b12e-92ef-459a-8318-b10e4368c6da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:56:06.651418 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9282b12e-92ef-459a-8318-b10e4368c6da admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:56:06.655034 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846edca3-c922-4bbf-8d25-46492a5cb6be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:56:06.656283 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846edca3-c922-4bbf-8d25-46492a5cb6be service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:56:06.656568 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846edca3-c922-4bbf-8d25-46492a5cb6be service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:56:06.656848 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846edca3-c922-4bbf-8d25-46492a5cb6be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:56:06.665701 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846edca3-c922-4bbf-8d25-46492a5cb6be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:56:06.667151 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846edca3-c922-4bbf-8d25-46492a5cb6be service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:56:06.676969 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1258/2526] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:56:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:56:07.124771 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab103ead-3a37-42c0-859e-3b121d497d34 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:56:07.126767 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-ab103ead-3a37-42c0-859e-3b121d497d34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:56:07.139494 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab103ead-3a37-42c0-859e-3b121d497d34 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:56:07.146594 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9173e5af-782a-4dc3-ae10-a2aaabee95ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:56:07.147525 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9173e5af-782a-4dc3-ae10-a2aaabee95ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:56:07.147946 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9173e5af-782a-4dc3-ae10-a2aaabee95ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:56:07.148236 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9173e5af-782a-4dc3-ae10-a2aaabee95ae admin admin] PATH_INFO: `/v3/projects/0fac226216fc494fb6452d619e5db30d` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:56:07.149447 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9173e5af-782a-4dc3-ae10-a2aaabee95ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=0fac226216fc494fb6452d619e5db30d)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:56:07.151677 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9173e5af-782a-4dc3-ae10-a2aaabee95ae admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:56:07.267476 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-9173e5af-782a-4dc3-ae10-a2aaabee95ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0fac226216fc494fb6452d619e5db30d', 'request_id': 'req-9173e5af-782a-4dc3-ae10-a2aaabee95ae'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:56:07.273742 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-9173e5af-782a-4dc3-ae10-a2aaabee95ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0fac226216fc494fb6452d619e5db30d', 'request_id': 'req-9173e5af-782a-4dc3-ae10-a2aaabee95ae'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:56:07.345790 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-9173e5af-782a-4dc3-ae10-a2aaabee95ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0fac226216fc494fb6452d619e5db30d', 'request_id': 'req-9173e5af-782a-4dc3-ae10-a2aaabee95ae'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:56:07.356815 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-9173e5af-782a-4dc3-ae10-a2aaabee95ae admin admin] The token cache is being invalidate because project 0fac226216fc494fb6452d619e5db30d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:56:07.361418 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-9173e5af-782a-4dc3-ae10-a2aaabee95ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:56:07.361418 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-9173e5af-782a-4dc3-ae10-a2aaabee95ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:56:07.361418 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1269/2527] 10.222.0.81 () {58 vars in 1232 bytes} [Wed Jul 20 22:56:07 2022] DELETE /identity/v3/projects/0fac226216fc494fb6452d619e5db30d => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:56:07.363886 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846edca3-c922-4bbf-8d25-46492a5cb6be service neutron] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:56:07.373868 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846edca3-c922-4bbf-8d25-46492a5cb6be service neutron] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:56:07.383771 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:56:07.432734 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23aeca0a-6fb8-47fa-ae5f-b4d159d2aac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:56:07.433764 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aeca0a-6fb8-47fa-ae5f-b4d159d2aac8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:56:07.434056 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aeca0a-6fb8-47fa-ae5f-b4d159d2aac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:56:07.434340 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aeca0a-6fb8-47fa-ae5f-b4d159d2aac8 admin admin] PATH_INFO: `/v3/users/b031a699050b4bfebe8170fe39874673` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:56:07.435621 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23aeca0a-6fb8-47fa-ae5f-b4d159d2aac8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b031a699050b4bfebe8170fe39874673)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:56:07.438266 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23aeca0a-6fb8-47fa-ae5f-b4d159d2aac8 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:56:07.571402 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-23aeca0a-6fb8-47fa-ae5f-b4d159d2aac8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b031a699050b4bfebe8170fe39874673'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:56:07.572057 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-23aeca0a-6fb8-47fa-ae5f-b4d159d2aac8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b031a699050b4bfebe8170fe39874673'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:56:07.572701 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-23aeca0a-6fb8-47fa-ae5f-b4d159d2aac8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b031a699050b4bfebe8170fe39874673'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:56:07.579619 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-23aeca0a-6fb8-47fa-ae5f-b4d159d2aac8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b031a699050b4bfebe8170fe39874673'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:56:07.626015 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-23aeca0a-6fb8-47fa-ae5f-b4d159d2aac8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b031a699050b4bfebe8170fe39874673'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:56:07.647294 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1259/2528] 10.222.0.81 () {58 vars in 1223 bytes} [Wed Jul 20 22:56:07 2022] DELETE /identity/v3/users/b031a699050b4bfebe8170fe39874673 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 22:56:08.016842 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9173e5af-782a-4dc3-ae10-a2aaabee95ae admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:56:08.018430 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-9173e5af-782a-4dc3-ae10-a2aaabee95ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:56:08.034515 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9173e5af-782a-4dc3-ae10-a2aaabee95ae admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:56:08.096482 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22ffc43-6d7c-4c64-95bb-312fe6b04407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:56:08.097371 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22ffc43-6d7c-4c64-95bb-312fe6b04407 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:56:08.097663 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22ffc43-6d7c-4c64-95bb-312fe6b04407 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:56:08.097933 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22ffc43-6d7c-4c64-95bb-312fe6b04407 admin admin] PATH_INFO: `/v3/projects/48504ae2828047a1baf5dfde5be82d00` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:56:08.099145 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22ffc43-6d7c-4c64-95bb-312fe6b04407 admin admin] RBAC: Authorizing `identity:delete_project(project_id=48504ae2828047a1baf5dfde5be82d00)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:56:08.101055 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22ffc43-6d7c-4c64-95bb-312fe6b04407 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:56:08.179458 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:56:08.182828 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-b22ffc43-6d7c-4c64-95bb-312fe6b04407 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48504ae2828047a1baf5dfde5be82d00', 'request_id': 'req-b22ffc43-6d7c-4c64-95bb-312fe6b04407'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:56:08.189438 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-b22ffc43-6d7c-4c64-95bb-312fe6b04407 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48504ae2828047a1baf5dfde5be82d00', 'request_id': 'req-b22ffc43-6d7c-4c64-95bb-312fe6b04407'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:56:08.236950 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-b22ffc43-6d7c-4c64-95bb-312fe6b04407 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48504ae2828047a1baf5dfde5be82d00', 'request_id': 'req-b22ffc43-6d7c-4c64-95bb-312fe6b04407'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:56:08.247348 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-b22ffc43-6d7c-4c64-95bb-312fe6b04407 admin admin] The token cache is being invalidate because project 48504ae2828047a1baf5dfde5be82d00 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:56:08.250182 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-b22ffc43-6d7c-4c64-95bb-312fe6b04407 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:56:08.250182 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-b22ffc43-6d7c-4c64-95bb-312fe6b04407 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:56:08.250620 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1270/2529] 10.222.0.81 () {58 vars in 1232 bytes} [Wed Jul 20 22:56:08 2022] DELETE /identity/v3/projects/48504ae2828047a1baf5dfde5be82d00 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:56:08.255385 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912e261c-7f7a-4f8f-bd46-f203fa95f71a None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:56:08.255682 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912e261c-7f7a-4f8f-bd46-f203fa95f71a None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:56:08.255971 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912e261c-7f7a-4f8f-bd46-f203fa95f71a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:56:08.274666 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-912e261c-7f7a-4f8f-bd46-f203fa95f71a None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:56:08.275921 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-912e261c-7f7a-4f8f-bd46-f203fa95f71a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:56:08.333647 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1260/2530] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:56:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:56:08.341487 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada73a28-90f7-4bfd-8a58-e604524c9a48 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:56:08.341780 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada73a28-90f7-4bfd-8a58-e604524c9a48 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:56:08.342061 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada73a28-90f7-4bfd-8a58-e604524c9a48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:56:08.362923 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-ada73a28-90f7-4bfd-8a58-e604524c9a48 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:56:08.364185 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-ada73a28-90f7-4bfd-8a58-e604524c9a48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:56:08.372262 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1271/2531] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:56:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:56:08.377307 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912e261c-7f7a-4f8f-bd46-f203fa95f71a None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:56:08.387296 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912e261c-7f7a-4f8f-bd46-f203fa95f71a None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:56:08.392414 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5a87f1-ce5a-4b0e-aca9-f38327e9895a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:56:08.393070 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5a87f1-ce5a-4b0e-aca9-f38327e9895a admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:56:08.393357 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5a87f1-ce5a-4b0e-aca9-f38327e9895a admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:56:08.393637 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5a87f1-ce5a-4b0e-aca9-f38327e9895a admin admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:56:08.394356 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5a87f1-ce5a-4b0e-aca9-f38327e9895a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:56:08.394666 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5a87f1-ce5a-4b0e-aca9-f38327e9895a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:56:08.396304 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5a87f1-ce5a-4b0e-aca9-f38327e9895a admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:56:08.408367 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1261/2532] 10.222.0.81 () {58 vars in 1152 bytes} [Wed Jul 20 22:56:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:56:08.412621 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada73a28-90f7-4bfd-8a58-e604524c9a48 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:56:08.421789 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada73a28-90f7-4bfd-8a58-e604524c9a48 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:56:08.426401 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8c8817-b313-49cc-a66a-44778932586f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:56:08.427081 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8c8817-b313-49cc-a66a-44778932586f admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:56:08.427368 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8c8817-b313-49cc-a66a-44778932586f admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:56:08.427644 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8c8817-b313-49cc-a66a-44778932586f admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:56:08.428320 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8c8817-b313-49cc-a66a-44778932586f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:56:08.430311 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8c8817-b313-49cc-a66a-44778932586f admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:56:08.518477 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1272/2533] 10.222.0.81 () {60 vars in 1152 bytes} [Wed Jul 20 22:56:08 2022] POST /identity/v3/projects => generated 381 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:56:08.522776 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5a87f1-ce5a-4b0e-aca9-f38327e9895a admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:56:08.532676 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5a87f1-ce5a-4b0e-aca9-f38327e9895a admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:56:08.549368 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:56:08.589296 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cb04b3-97d8-4deb-8648-9b4b8d5d64dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:56:08.589972 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cb04b3-97d8-4deb-8648-9b4b8d5d64dd admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:56:08.590252 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cb04b3-97d8-4deb-8648-9b4b8d5d64dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:56:08.590525 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cb04b3-97d8-4deb-8648-9b4b8d5d64dd admin admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:56:08.591237 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cb04b3-97d8-4deb-8648-9b4b8d5d64dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:56:08.593429 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cb04b3-97d8-4deb-8648-9b4b8d5d64dd admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:56:08.683425 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1262/2534] 10.222.0.81 () {60 vars in 1143 bytes} [Wed Jul 20 22:56:08 2022] POST /identity/v3/users => generated 291 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:56:08.687669 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8c8817-b313-49cc-a66a-44778932586f admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:56:08.698276 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8c8817-b313-49cc-a66a-44778932586f admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:56:08.703786 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9857bb4b-2418-473b-bd36-544e9e9b7a03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:56:08.704625 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9857bb4b-2418-473b-bd36-544e9e9b7a03 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:56:08.704919 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9857bb4b-2418-473b-bd36-544e9e9b7a03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:56:08.705211 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9857bb4b-2418-473b-bd36-544e9e9b7a03 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:56:08.705936 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9857bb4b-2418-473b-bd36-544e9e9b7a03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:56:08.707585 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9857bb4b-2418-473b-bd36-544e9e9b7a03 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:56:08.718965 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1273/2535] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:56:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:56:08.723100 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cb04b3-97d8-4deb-8648-9b4b8d5d64dd admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:56:08.735088 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9857bb4b-2418-473b-bd36-544e9e9b7a03 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:56:08.735325 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cb04b3-97d8-4deb-8648-9b4b8d5d64dd admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:56:08.736975 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-9857bb4b-2418-473b-bd36-544e9e9b7a03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:56:08.740335 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe47682-f21c-49f1-ac1d-12c3886c55a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:56:08.741049 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe47682-f21c-49f1-ac1d-12c3886c55a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:56:08.741490 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe47682-f21c-49f1-ac1d-12c3886c55a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:56:08.741805 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe47682-f21c-49f1-ac1d-12c3886c55a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:56:08.742538 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe47682-f21c-49f1-ac1d-12c3886c55a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:56:08.745121 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe47682-f21c-49f1-ac1d-12c3886c55a9 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:56:08.756100 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1263/2536] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:56:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:56:08.760840 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe47682-f21c-49f1-ac1d-12c3886c55a9 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:56:08.772576 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe47682-f21c-49f1-ac1d-12c3886c55a9 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:56:08.777632 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cc2feb-1a1c-4fe6-97b3-d1b47346ec0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:56:08.778524 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cc2feb-1a1c-4fe6-97b3-d1b47346ec0d admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:56:08.778815 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cc2feb-1a1c-4fe6-97b3-d1b47346ec0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:56:08.779611 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cc2feb-1a1c-4fe6-97b3-d1b47346ec0d admin admin] PATH_INFO: `/v3/projects/1278cca4c1124564b83a914229ceddd0/users/cebefefe697a4e978d520f24eb71c949/roles/559942c4de1744169f8b5e0404ecde6b` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:56:08.795933 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cc2feb-1a1c-4fe6-97b3-d1b47346ec0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1278cca4c1124564b83a914229ceddd0, user_id=cebefefe697a4e978d520f24eb71c949, role_id=559942c4de1744169f8b5e0404ecde6b)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:56:08.796570 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9857bb4b-2418-473b-bd36-544e9e9b7a03 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:56:08.798624 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cc2feb-1a1c-4fe6-97b3-d1b47346ec0d admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:56:08.800472 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a922af95-b36d-42f2-b932-b5f64d5b5c1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:56:08.802538 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a922af95-b36d-42f2-b932-b5f64d5b5c1a service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:56:08.802538 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a922af95-b36d-42f2-b932-b5f64d5b5c1a service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:56:08.802538 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a922af95-b36d-42f2-b932-b5f64d5b5c1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:56:08.803663 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-a922af95-b36d-42f2-b932-b5f64d5b5c1a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:56:08.814454 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a922af95-b36d-42f2-b932-b5f64d5b5c1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:56:08.815913 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a922af95-b36d-42f2-b932-b5f64d5b5c1a service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:56:08.824037 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1274/2537] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:56:08 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:56:08.854837 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1264/2538] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:56:08 2022] PUT /identity/v3/projects/1278cca4c1124564b83a914229ceddd0/users/cebefefe697a4e978d520f24eb71c949/roles/559942c4de1744169f8b5e0404ecde6b => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:56:08.865966 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f2a83d-c41e-45b0-9934-ae1d2f984c74 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:56:08.866266 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f2a83d-c41e-45b0-9934-ae1d2f984c74 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:56:08.866553 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f2a83d-c41e-45b0-9934-ae1d2f984c74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:56:08.893135 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-81f2a83d-c41e-45b0-9934-ae1d2f984c74 None None] MFA Rules not processed for user `cebefefe697a4e978d520f24eb71c949`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:56:08.894347 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-81f2a83d-c41e-45b0-9934-ae1d2f984c74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:56:08.957639 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1275/2539] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:56:08 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:56:08.962069 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cc2feb-1a1c-4fe6-97b3-d1b47346ec0d admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:56:09.024165 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cc2feb-1a1c-4fe6-97b3-d1b47346ec0d admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:56:09.029038 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf4c97d-b969-41d0-857f-888c38979937 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:56:09.029713 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf4c97d-b969-41d0-857f-888c38979937 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:56:09.029997 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf4c97d-b969-41d0-857f-888c38979937 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:56:09.030304 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf4c97d-b969-41d0-857f-888c38979937 admin admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:56:09.031097 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf4c97d-b969-41d0-857f-888c38979937 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:56:09.033029 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf4c97d-b969-41d0-857f-888c38979937 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:56:09.130022 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1265/2540] 10.222.0.81 () {60 vars in 1152 bytes} [Wed Jul 20 22:56:08 2022] POST /identity/v3/projects => generated 379 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:56:09.134076 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f2a83d-c41e-45b0-9934-ae1d2f984c74 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:56:09.145208 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f2a83d-c41e-45b0-9934-ae1d2f984c74 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:56:09.170528 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:56:09.201980 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87faeeee-d712-45b9-9b9e-639d8fe75936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:56:09.202684 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87faeeee-d712-45b9-9b9e-639d8fe75936 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:56:09.202974 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87faeeee-d712-45b9-9b9e-639d8fe75936 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:56:09.203292 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87faeeee-d712-45b9-9b9e-639d8fe75936 admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:56:09.204034 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87faeeee-d712-45b9-9b9e-639d8fe75936 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:56:09.205999 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87faeeee-d712-45b9-9b9e-639d8fe75936 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:56:09.273847 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1276/2541] 10.222.0.81 () {60 vars in 1143 bytes} [Wed Jul 20 22:56:09 2022] POST /identity/v3/users => generated 296 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:56:09.275145 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf4c97d-b969-41d0-857f-888c38979937 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:56:09.287619 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf4c97d-b969-41d0-857f-888c38979937 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:56:09.292232 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b4dfed-9bbc-4cb4-b25c-792ba1fca219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:56:09.292930 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b4dfed-9bbc-4cb4-b25c-792ba1fca219 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:56:09.293217 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b4dfed-9bbc-4cb4-b25c-792ba1fca219 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:56:09.293497 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b4dfed-9bbc-4cb4-b25c-792ba1fca219 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:56:09.294248 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b4dfed-9bbc-4cb4-b25c-792ba1fca219 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:56:09.295892 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b4dfed-9bbc-4cb4-b25c-792ba1fca219 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:56:09.307539 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1266/2542] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:56:09 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 22:56:09.311851 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87faeeee-d712-45b9-9b9e-639d8fe75936 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:56:09.321767 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87faeeee-d712-45b9-9b9e-639d8fe75936 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:56:09.326317 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ff701b-2dd7-44f0-b2cc-818751823ee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:56:09.327249 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ff701b-2dd7-44f0-b2cc-818751823ee9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:56:09.327538 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ff701b-2dd7-44f0-b2cc-818751823ee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:56:09.327821 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ff701b-2dd7-44f0-b2cc-818751823ee9 admin admin] PATH_INFO: `/v3/projects/9440d5c4ebf943a9b7b782b79e29e776/users/12024fbb91194767bead476e8a74896a/roles/d89b6c92d35a401a8517f1d47f6be843` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:56:09.342710 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ff701b-2dd7-44f0-b2cc-818751823ee9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9440d5c4ebf943a9b7b782b79e29e776, user_id=12024fbb91194767bead476e8a74896a, role_id=d89b6c92d35a401a8517f1d47f6be843)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:56:09.345917 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ff701b-2dd7-44f0-b2cc-818751823ee9 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:56:09.425785 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1277/2543] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:56:09 2022] PUT /identity/v3/projects/9440d5c4ebf943a9b7b782b79e29e776/users/12024fbb91194767bead476e8a74896a/roles/d89b6c92d35a401a8517f1d47f6be843 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:56:09.429742 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b4dfed-9bbc-4cb4-b25c-792ba1fca219 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:56:09.440421 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b4dfed-9bbc-4cb4-b25c-792ba1fca219 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:56:09.485548 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:56:09.497090 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0acff3-1476-4a5b-a1cc-745f52fa18b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:56:09.500867 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0acff3-1476-4a5b-a1cc-745f52fa18b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:56:09.500867 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0acff3-1476-4a5b-a1cc-745f52fa18b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:56:09.500867 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0acff3-1476-4a5b-a1cc-745f52fa18b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:56:09.500867 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0acff3-1476-4a5b-a1cc-745f52fa18b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:56:09.501307 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0acff3-1476-4a5b-a1cc-745f52fa18b8 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:56:09.511221 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1267/2544] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:56:09 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:56:09.516255 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ff701b-2dd7-44f0-b2cc-818751823ee9 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:56:09.526507 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ff701b-2dd7-44f0-b2cc-818751823ee9 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:56:09.531113 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e03213e-a906-485b-a9ac-9599d8589e51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:56:09.535303 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e03213e-a906-485b-a9ac-9599d8589e51 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:56:09.535303 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e03213e-a906-485b-a9ac-9599d8589e51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:56:09.535303 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e03213e-a906-485b-a9ac-9599d8589e51 admin admin] PATH_INFO: `/v3/projects/9440d5c4ebf943a9b7b782b79e29e776/users/12024fbb91194767bead476e8a74896a/roles/d89b6c92d35a401a8517f1d47f6be843` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:56:09.535303 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e03213e-a906-485b-a9ac-9599d8589e51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9440d5c4ebf943a9b7b782b79e29e776, user_id=12024fbb91194767bead476e8a74896a, role_id=d89b6c92d35a401a8517f1d47f6be843)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:56:09.537152 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e03213e-a906-485b-a9ac-9599d8589e51 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:56:09.547138 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1278/2545] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:56:09 2022] PUT /identity/v3/projects/9440d5c4ebf943a9b7b782b79e29e776/users/12024fbb91194767bead476e8a74896a/roles/d89b6c92d35a401a8517f1d47f6be843 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:56:09.558197 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71baff13-a8f8-430c-a7e1-29b2d86304d1 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:56:09.558492 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71baff13-a8f8-430c-a7e1-29b2d86304d1 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:56:09.558806 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71baff13-a8f8-430c-a7e1-29b2d86304d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:56:09.585691 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-71baff13-a8f8-430c-a7e1-29b2d86304d1 None None] MFA Rules not processed for user `12024fbb91194767bead476e8a74896a`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:56:09.586914 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-71baff13-a8f8-430c-a7e1-29b2d86304d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:56:09.655286 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1268/2546] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:56:09 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 22:56:09.666132 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e03213e-a906-485b-a9ac-9599d8589e51 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:56:09.667859 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-3e03213e-a906-485b-a9ac-9599d8589e51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:56:09.725299 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e03213e-a906-485b-a9ac-9599d8589e51 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:56:09.729009 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7ba184-74a1-4908-a37a-9d4ac8cfd46c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecf4db289fba4e4a806198c62d31f7c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:56:09.729655 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7ba184-74a1-4908-a37a-9d4ac8cfd46c service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:56:09.729947 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7ba184-74a1-4908-a37a-9d4ac8cfd46c service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:56:09.730232 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7ba184-74a1-4908-a37a-9d4ac8cfd46c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:56:09.740296 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7ba184-74a1-4908-a37a-9d4ac8cfd46c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:56:09.741668 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7ba184-74a1-4908-a37a-9d4ac8cfd46c service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:56:09.752276 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1279/2547] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:56:09 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:56:10.256933 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71baff13-a8f8-430c-a7e1-29b2d86304d1 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:56:10.320703 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71baff13-a8f8-430c-a7e1-29b2d86304d1 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:56:10.325246 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1537b7cc-7a6c-4db0-a3f8-ccbd2f7cbb9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:56:10.326133 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1537b7cc-7a6c-4db0-a3f8-ccbd2f7cbb9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:56:10.326423 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1537b7cc-7a6c-4db0-a3f8-ccbd2f7cbb9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:56:10.326701 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1537b7cc-7a6c-4db0-a3f8-ccbd2f7cbb9f admin admin] PATH_INFO: `/v3/users/cebefefe697a4e978d520f24eb71c949` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:56:10.330770 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1537b7cc-7a6c-4db0-a3f8-ccbd2f7cbb9f admin admin] RBAC: Authorizing `identity:delete_user(user_id=cebefefe697a4e978d520f24eb71c949)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:56:10.335386 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1537b7cc-7a6c-4db0-a3f8-ccbd2f7cbb9f admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:56:10.521098 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-1537b7cc-7a6c-4db0-a3f8-ccbd2f7cbb9f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cebefefe697a4e978d520f24eb71c949'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:56:10.521767 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-1537b7cc-7a6c-4db0-a3f8-ccbd2f7cbb9f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cebefefe697a4e978d520f24eb71c949'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:56:10.522429 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-1537b7cc-7a6c-4db0-a3f8-ccbd2f7cbb9f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cebefefe697a4e978d520f24eb71c949'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:56:10.529476 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-1537b7cc-7a6c-4db0-a3f8-ccbd2f7cbb9f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cebefefe697a4e978d520f24eb71c949'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:56:10.596703 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-1537b7cc-7a6c-4db0-a3f8-ccbd2f7cbb9f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cebefefe697a4e978d520f24eb71c949'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:56:10.604475 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:56:10.615004 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1269/2548] 10.222.0.81 () {58 vars in 1223 bytes} [Wed Jul 20 22:56:10 2022] DELETE /identity/v3/users/cebefefe697a4e978d520f24eb71c949 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:56:10.619569 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d18634-9b9b-4542-a9a5-d2c60a8bd2de None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:56:10.620028 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d18634-9b9b-4542-a9a5-d2c60a8bd2de None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:56:10.620139 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d18634-9b9b-4542-a9a5-d2c60a8bd2de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:56:10.637549 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-c7d18634-9b9b-4542-a9a5-d2c60a8bd2de None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:56:10.638745 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-c7d18634-9b9b-4542-a9a5-d2c60a8bd2de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:56:10.696926 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1280/2549] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:56:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:56:10.707310 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1537b7cc-7a6c-4db0-a3f8-ccbd2f7cbb9f admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:56:10.708745 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-1537b7cc-7a6c-4db0-a3f8-ccbd2f7cbb9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:56:10.767545 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1537b7cc-7a6c-4db0-a3f8-ccbd2f7cbb9f admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:56:10.772067 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5521255-4193-4ae2-b3bc-fe86282ae522 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:56:10.772660 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5521255-4193-4ae2-b3bc-fe86282ae522 service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:56:10.772946 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5521255-4193-4ae2-b3bc-fe86282ae522 service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:56:10.773224 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5521255-4193-4ae2-b3bc-fe86282ae522 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:56:10.782325 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5521255-4193-4ae2-b3bc-fe86282ae522 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:56:10.783871 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5521255-4193-4ae2-b3bc-fe86282ae522 service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:56:10.793301 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1270/2550] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:56:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:56:11.250272 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d18634-9b9b-4542-a9a5-d2c60a8bd2de None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:56:11.251818 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-c7d18634-9b9b-4542-a9a5-d2c60a8bd2de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:56:11.264739 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d18634-9b9b-4542-a9a5-d2c60a8bd2de None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:56:11.269889 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74e8634-b42d-4d72-902d-afac979a6d6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:56:11.270882 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74e8634-b42d-4d72-902d-afac979a6d6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:56:11.271243 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74e8634-b42d-4d72-902d-afac979a6d6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:56:11.271693 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74e8634-b42d-4d72-902d-afac979a6d6f admin admin] PATH_INFO: `/v3/projects/1278cca4c1124564b83a914229ceddd0` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:56:11.272955 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74e8634-b42d-4d72-902d-afac979a6d6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=1278cca4c1124564b83a914229ceddd0)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:56:11.274950 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74e8634-b42d-4d72-902d-afac979a6d6f admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:56:11.297157 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:56:11.385544 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-f74e8634-b42d-4d72-902d-afac979a6d6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1278cca4c1124564b83a914229ceddd0', 'request_id': 'req-f74e8634-b42d-4d72-902d-afac979a6d6f'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:56:11.391929 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-f74e8634-b42d-4d72-902d-afac979a6d6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1278cca4c1124564b83a914229ceddd0', 'request_id': 'req-f74e8634-b42d-4d72-902d-afac979a6d6f'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:56:11.471494 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-f74e8634-b42d-4d72-902d-afac979a6d6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1278cca4c1124564b83a914229ceddd0', 'request_id': 'req-f74e8634-b42d-4d72-902d-afac979a6d6f'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:56:11.482798 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-f74e8634-b42d-4d72-902d-afac979a6d6f admin admin] The token cache is being invalidate because project 1278cca4c1124564b83a914229ceddd0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:56:11.483122 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-f74e8634-b42d-4d72-902d-afac979a6d6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:56:11.484284 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-f74e8634-b42d-4d72-902d-afac979a6d6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:56:11.485864 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1281/2551] 10.222.0.81 () {58 vars in 1232 bytes} [Wed Jul 20 22:56:11 2022] DELETE /identity/v3/projects/1278cca4c1124564b83a914229ceddd0 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:56:11.490110 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5521255-4193-4ae2-b3bc-fe86282ae522 service neutron] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:56:11.500570 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5521255-4193-4ae2-b3bc-fe86282ae522 service neutron] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:56:11.559401 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5c06ac-6618-4119-965c-636dc0528ac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:56:11.560294 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5c06ac-6618-4119-965c-636dc0528ac0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:56:11.560586 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5c06ac-6618-4119-965c-636dc0528ac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:56:11.560866 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5c06ac-6618-4119-965c-636dc0528ac0 admin admin] PATH_INFO: `/v3/users/12024fbb91194767bead476e8a74896a` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:56:11.562085 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5c06ac-6618-4119-965c-636dc0528ac0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=12024fbb91194767bead476e8a74896a)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:56:11.564055 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5c06ac-6618-4119-965c-636dc0528ac0 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:56:11.747853 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-8d5c06ac-6618-4119-965c-636dc0528ac0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12024fbb91194767bead476e8a74896a'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:56:11.748515 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-8d5c06ac-6618-4119-965c-636dc0528ac0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12024fbb91194767bead476e8a74896a'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:56:11.749147 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-8d5c06ac-6618-4119-965c-636dc0528ac0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12024fbb91194767bead476e8a74896a'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:56:11.756158 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-8d5c06ac-6618-4119-965c-636dc0528ac0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12024fbb91194767bead476e8a74896a'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:56:11.830913 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-8d5c06ac-6618-4119-965c-636dc0528ac0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12024fbb91194767bead476e8a74896a'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:56:11.848987 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1271/2552] 10.222.0.81 () {58 vars in 1223 bytes} [Wed Jul 20 22:56:11 2022] DELETE /identity/v3/users/12024fbb91194767bead476e8a74896a => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:56:12.274614 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74e8634-b42d-4d72-902d-afac979a6d6f admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:56:12.276043 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-f74e8634-b42d-4d72-902d-afac979a6d6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:56:12.289814 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74e8634-b42d-4d72-902d-afac979a6d6f admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:56:12.350245 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2555a849-72f5-42dd-9af6-839606b3bda3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:56:12.351088 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2555a849-72f5-42dd-9af6-839606b3bda3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:56:12.351380 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2555a849-72f5-42dd-9af6-839606b3bda3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:56:12.351661 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2555a849-72f5-42dd-9af6-839606b3bda3 admin admin] PATH_INFO: `/v3/projects/9440d5c4ebf943a9b7b782b79e29e776` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:56:12.352845 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2555a849-72f5-42dd-9af6-839606b3bda3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9440d5c4ebf943a9b7b782b79e29e776)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:56:12.354808 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2555a849-72f5-42dd-9af6-839606b3bda3 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:56:12.460346 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-2555a849-72f5-42dd-9af6-839606b3bda3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9440d5c4ebf943a9b7b782b79e29e776', 'request_id': 'req-2555a849-72f5-42dd-9af6-839606b3bda3'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:56:12.466508 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-2555a849-72f5-42dd-9af6-839606b3bda3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9440d5c4ebf943a9b7b782b79e29e776', 'request_id': 'req-2555a849-72f5-42dd-9af6-839606b3bda3'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:56:12.591684 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-2555a849-72f5-42dd-9af6-839606b3bda3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9440d5c4ebf943a9b7b782b79e29e776', 'request_id': 'req-2555a849-72f5-42dd-9af6-839606b3bda3'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:56:12.602931 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-2555a849-72f5-42dd-9af6-839606b3bda3 admin admin] The token cache is being invalidate because project 9440d5c4ebf943a9b7b782b79e29e776 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:56:12.603196 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-2555a849-72f5-42dd-9af6-839606b3bda3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:56:12.603865 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-2555a849-72f5-42dd-9af6-839606b3bda3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:56:12.605418 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1282/2553] 10.222.0.81 () {58 vars in 1232 bytes} [Wed Jul 20 22:56:12 2022] DELETE /identity/v3/projects/9440d5c4ebf943a9b7b782b79e29e776 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:56:12.610975 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6310c53-cca4-4062-940f-1b606ce34fd9 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:56:12.611484 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6310c53-cca4-4062-940f-1b606ce34fd9 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:56:12.611580 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6310c53-cca4-4062-940f-1b606ce34fd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:56:12.630192 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-d6310c53-cca4-4062-940f-1b606ce34fd9 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:56:12.631444 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-d6310c53-cca4-4062-940f-1b606ce34fd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:56:12.636961 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:56:12.691347 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1272/2554] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:56:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:56:12.699157 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceb9a99-d082-4013-aa23-cb3b40b42fae None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:56:12.703828 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceb9a99-d082-4013-aa23-cb3b40b42fae None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:56:12.703828 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceb9a99-d082-4013-aa23-cb3b40b42fae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:56:12.719020 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-5ceb9a99-d082-4013-aa23-cb3b40b42fae None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:56:12.720250 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-5ceb9a99-d082-4013-aa23-cb3b40b42fae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:56:12.728193 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1283/2555] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:56:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:56:12.732291 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6310c53-cca4-4062-940f-1b606ce34fd9 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:56:12.742632 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6310c53-cca4-4062-940f-1b606ce34fd9 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:56:12.747928 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989db8ef-8e42-4e5d-b315-dbbe47756561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:56:12.748602 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989db8ef-8e42-4e5d-b315-dbbe47756561 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:56:12.748891 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989db8ef-8e42-4e5d-b315-dbbe47756561 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:56:12.749232 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989db8ef-8e42-4e5d-b315-dbbe47756561 admin admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:56:12.750541 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989db8ef-8e42-4e5d-b315-dbbe47756561 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:56:12.751374 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989db8ef-8e42-4e5d-b315-dbbe47756561 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:56:12.753496 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989db8ef-8e42-4e5d-b315-dbbe47756561 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:56:12.765863 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1273/2556] 10.222.0.81 () {58 vars in 1152 bytes} [Wed Jul 20 22:56:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:56:12.770269 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ceb9a99-d082-4013-aa23-cb3b40b42fae None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:56:12.779851 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ceb9a99-d082-4013-aa23-cb3b40b42fae None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:56:12.785011 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c6e408-48ca-4810-9403-045463d41c79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:56:12.785692 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c6e408-48ca-4810-9403-045463d41c79 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:56:12.785978 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c6e408-48ca-4810-9403-045463d41c79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:56:12.786257 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c6e408-48ca-4810-9403-045463d41c79 admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:56:12.790090 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c6e408-48ca-4810-9403-045463d41c79 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:56:12.790090 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c6e408-48ca-4810-9403-045463d41c79 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:56:12.907649 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1284/2557] 10.222.0.81 () {60 vars in 1152 bytes} [Wed Jul 20 22:56:12 2022] POST /identity/v3/projects => generated 385 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:56:12.911903 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989db8ef-8e42-4e5d-b315-dbbe47756561 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:56:12.922464 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989db8ef-8e42-4e5d-b315-dbbe47756561 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:56:12.978186 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd53cad1-e52e-4fa3-aae0-96cd40ed5e88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:56:12.978865 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd53cad1-e52e-4fa3-aae0-96cd40ed5e88 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:56:12.979192 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd53cad1-e52e-4fa3-aae0-96cd40ed5e88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:56:12.979479 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd53cad1-e52e-4fa3-aae0-96cd40ed5e88 admin admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:56:12.980171 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd53cad1-e52e-4fa3-aae0-96cd40ed5e88 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:56:12.982096 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd53cad1-e52e-4fa3-aae0-96cd40ed5e88 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:56:13.077295 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1274/2558] 10.222.0.81 () {60 vars in 1143 bytes} [Wed Jul 20 22:56:12 2022] POST /identity/v3/users => generated 293 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 22:56:13.081650 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c6e408-48ca-4810-9403-045463d41c79 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:56:13.091935 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c6e408-48ca-4810-9403-045463d41c79 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:56:13.097166 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374d78b4-102a-4efb-87a2-335e5a02f8aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:56:13.098370 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374d78b4-102a-4efb-87a2-335e5a02f8aa admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:56:13.098370 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374d78b4-102a-4efb-87a2-335e5a02f8aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:56:13.098370 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374d78b4-102a-4efb-87a2-335e5a02f8aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:56:13.099117 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374d78b4-102a-4efb-87a2-335e5a02f8aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:56:13.100731 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374d78b4-102a-4efb-87a2-335e5a02f8aa admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:56:13.102749 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:56:13.112011 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1285/2559] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:56:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:56:13.115921 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd53cad1-e52e-4fa3-aae0-96cd40ed5e88 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:56:13.125979 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd53cad1-e52e-4fa3-aae0-96cd40ed5e88 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:56:13.130633 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834a2d8e-7dfe-4a88-a5cd-585c00bc73ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:56:13.131329 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834a2d8e-7dfe-4a88-a5cd-585c00bc73ed admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:56:13.131621 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834a2d8e-7dfe-4a88-a5cd-585c00bc73ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:56:13.131904 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834a2d8e-7dfe-4a88-a5cd-585c00bc73ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:56:13.132631 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834a2d8e-7dfe-4a88-a5cd-585c00bc73ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:56:13.134251 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834a2d8e-7dfe-4a88-a5cd-585c00bc73ed admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:56:13.144440 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1275/2560] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:56:13 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:56:13.148415 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374d78b4-102a-4efb-87a2-335e5a02f8aa admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:56:13.158813 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374d78b4-102a-4efb-87a2-335e5a02f8aa admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:56:13.163482 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd039e9-330f-4fdd-a420-f700db93880f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:56:13.167474 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd039e9-330f-4fdd-a420-f700db93880f admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:56:13.167474 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd039e9-330f-4fdd-a420-f700db93880f admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:56:13.167474 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd039e9-330f-4fdd-a420-f700db93880f admin admin] PATH_INFO: `/v3/projects/c1490850f95249eebdc9ae86de441023/users/798ac7e3c9d4496ba731d592c5326380/roles/559942c4de1744169f8b5e0404ecde6b` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:56:13.180014 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd039e9-330f-4fdd-a420-f700db93880f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1490850f95249eebdc9ae86de441023, user_id=798ac7e3c9d4496ba731d592c5326380, role_id=559942c4de1744169f8b5e0404ecde6b)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:56:13.182652 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd039e9-330f-4fdd-a420-f700db93880f admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:56:13.260132 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1286/2561] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:56:13 2022] PUT /identity/v3/projects/c1490850f95249eebdc9ae86de441023/users/798ac7e3c9d4496ba731d592c5326380/roles/559942c4de1744169f8b5e0404ecde6b => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:56:13.271567 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaafb989-7cce-4776-91e3-ef6c82beee9a None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:56:13.271864 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaafb989-7cce-4776-91e3-ef6c82beee9a None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:56:13.272150 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaafb989-7cce-4776-91e3-ef6c82beee9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:56:13.299421 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-eaafb989-7cce-4776-91e3-ef6c82beee9a None None] MFA Rules not processed for user `798ac7e3c9d4496ba731d592c5326380`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:56:13.300633 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-eaafb989-7cce-4776-91e3-ef6c82beee9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:56:13.366505 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1276/2562] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:56:13 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:56:13.377491 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd039e9-330f-4fdd-a420-f700db93880f admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:56:13.382423 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-1fd039e9-330f-4fdd-a420-f700db93880f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:56:13.444863 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd039e9-330f-4fdd-a420-f700db93880f admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:56:13.448559 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880009db-1779-4ea9-b313-8608ce399c67 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecf4db289fba4e4a806198c62d31f7c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:56:13.449225 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880009db-1779-4ea9-b313-8608ce399c67 service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:56:13.449514 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880009db-1779-4ea9-b313-8608ce399c67 service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:56:13.449797 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880009db-1779-4ea9-b313-8608ce399c67 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:56:13.459276 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880009db-1779-4ea9-b313-8608ce399c67 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:56:13.460712 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880009db-1779-4ea9-b313-8608ce399c67 service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:56:13.470604 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1287/2563] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:56:13 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:56:13.489429 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaafb989-7cce-4776-91e3-ef6c82beee9a None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:56:13.498222 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:56:13.553239 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaafb989-7cce-4776-91e3-ef6c82beee9a None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:56:13.557861 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75a7d8c-17d7-469c-a711-5ca8b6dcac1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:56:13.558773 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75a7d8c-17d7-469c-a711-5ca8b6dcac1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:56:13.559097 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75a7d8c-17d7-469c-a711-5ca8b6dcac1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:56:13.559381 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75a7d8c-17d7-469c-a711-5ca8b6dcac1c admin admin] PATH_INFO: `/v3/users/798ac7e3c9d4496ba731d592c5326380` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:56:13.560611 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75a7d8c-17d7-469c-a711-5ca8b6dcac1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=798ac7e3c9d4496ba731d592c5326380)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:56:13.563174 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75a7d8c-17d7-469c-a711-5ca8b6dcac1c admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:56:13.736638 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-e75a7d8c-17d7-469c-a711-5ca8b6dcac1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '798ac7e3c9d4496ba731d592c5326380'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:56:13.737285 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-e75a7d8c-17d7-469c-a711-5ca8b6dcac1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '798ac7e3c9d4496ba731d592c5326380'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:56:13.737830 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-e75a7d8c-17d7-469c-a711-5ca8b6dcac1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '798ac7e3c9d4496ba731d592c5326380'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:56:13.744857 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-e75a7d8c-17d7-469c-a711-5ca8b6dcac1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '798ac7e3c9d4496ba731d592c5326380'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:56:13.831421 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-e75a7d8c-17d7-469c-a711-5ca8b6dcac1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '798ac7e3c9d4496ba731d592c5326380'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:56:13.849882 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1277/2564] 10.222.0.81 () {58 vars in 1223 bytes} [Wed Jul 20 22:56:13 2022] DELETE /identity/v3/users/798ac7e3c9d4496ba731d592c5326380 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:56:13.854564 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23f9511-5c29-4637-8ea3-ae7f8e7333fe None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:56:13.854861 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23f9511-5c29-4637-8ea3-ae7f8e7333fe None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:56:13.855169 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23f9511-5c29-4637-8ea3-ae7f8e7333fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:56:13.872230 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-c23f9511-5c29-4637-8ea3-ae7f8e7333fe None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:56:13.873429 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-c23f9511-5c29-4637-8ea3-ae7f8e7333fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:56:13.931158 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1288/2565] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:56:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:56:13.940970 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75a7d8c-17d7-469c-a711-5ca8b6dcac1c admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:56:13.942376 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-e75a7d8c-17d7-469c-a711-5ca8b6dcac1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:56:14.000449 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75a7d8c-17d7-469c-a711-5ca8b6dcac1c admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:56:14.004176 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025d19f2-db7f-4a4f-bc33-15f33a59d77a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:56:14.004829 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025d19f2-db7f-4a4f-bc33-15f33a59d77a service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:56:14.005118 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025d19f2-db7f-4a4f-bc33-15f33a59d77a service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:56:14.005402 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025d19f2-db7f-4a4f-bc33-15f33a59d77a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:56:14.015217 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025d19f2-db7f-4a4f-bc33-15f33a59d77a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:56:14.016731 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025d19f2-db7f-4a4f-bc33-15f33a59d77a service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:56:14.026356 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1278/2566] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:56:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:56:14.469767 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfce24e5-cdf6-4024-80c2-3f39e1fbc039 None None] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:56:14.470285 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfce24e5-cdf6-4024-80c2-3f39e1fbc039 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:56:14.470392 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfce24e5-cdf6-4024-80c2-3f39e1fbc039 None None] PATH_INFO: `/` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:56:14.471941 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1289/2567] 10.222.0.81 () {54 vars in 921 bytes} [Wed Jul 20 22:56:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 22:56:14.479514 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cddc0d-5713-4a87-9fd1-4953dd1eb789 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:56:14.479514 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cddc0d-5713-4a87-9fd1-4953dd1eb789 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:56:14.479514 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cddc0d-5713-4a87-9fd1-4953dd1eb789 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:56:14.508033 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfce24e5-cdf6-4024-80c2-3f39e1fbc039 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:56:14.510733 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-dfce24e5-cdf6-4024-80c2-3f39e1fbc039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:56:14.526855 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-80cddc0d-5713-4a87-9fd1-4953dd1eb789 None None] MFA Rules not processed for user `333c2b1b11cc4c478abfd965a72e9d66`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:56:14.527976 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-80cddc0d-5713-4a87-9fd1-4953dd1eb789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:56:14.537182 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1279/2568] 10.222.0.81 () {58 vars in 1018 bytes} [Wed Jul 20 22:56:14 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:56:14.552727 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfce24e5-cdf6-4024-80c2-3f39e1fbc039 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:56:14.557724 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cddc0d-5713-4a87-9fd1-4953dd1eb789 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:56:14.558940 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ceaef2e-fafe-4394-bb1e-03d70d08e579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:56:14.559398 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-80cddc0d-5713-4a87-9fd1-4953dd1eb789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:56:14.560296 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceaef2e-fafe-4394-bb1e-03d70d08e579 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:56:14.560740 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceaef2e-fafe-4394-bb1e-03d70d08e579 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:56:14.561317 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceaef2e-fafe-4394-bb1e-03d70d08e579 admin admin] PATH_INFO: `/v3/projects/c1490850f95249eebdc9ae86de441023` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:56:14.562993 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ceaef2e-fafe-4394-bb1e-03d70d08e579 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1490850f95249eebdc9ae86de441023)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:56:14.565493 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ceaef2e-fafe-4394-bb1e-03d70d08e579 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:56:14.574200 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cddc0d-5713-4a87-9fd1-4953dd1eb789 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:56:14.578460 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb523baa-eea9-4aab-8845-842486f78dce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:56:14.579302 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb523baa-eea9-4aab-8845-842486f78dce service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:56:14.579677 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb523baa-eea9-4aab-8845-842486f78dce service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:56:14.580055 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb523baa-eea9-4aab-8845-842486f78dce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:56:14.610916 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb523baa-eea9-4aab-8845-842486f78dce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:56:14.612577 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb523baa-eea9-4aab-8845-842486f78dce service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:56:14.622110 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1280/2569] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:56:14 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:56:14.738874 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8965e2e-b00e-423c-8925-f759cfd22084 None None] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:56:14.739480 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8965e2e-b00e-423c-8925-f759cfd22084 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:56:14.739480 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8965e2e-b00e-423c-8925-f759cfd22084 None None] PATH_INFO: `/` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:56:14.740846 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1281/2570] 10.222.0.81 () {54 vars in 921 bytes} [Wed Jul 20 22:56:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 22:56:14.745714 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc8c336-6e2c-49d9-81e9-24b91ba9e877 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:56:14.746252 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc8c336-6e2c-49d9-81e9-24b91ba9e877 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:56:14.746616 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc8c336-6e2c-49d9-81e9-24b91ba9e877 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:56:14.768736 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-ddc8c336-6e2c-49d9-81e9-24b91ba9e877 None None] MFA Rules not processed for user `7829324db2574d38b18c749d0559177f`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:56:14.769944 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-ddc8c336-6e2c-49d9-81e9-24b91ba9e877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:56:14.776450 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:56:14.783836 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-6ceaef2e-fafe-4394-bb1e-03d70d08e579 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1490850f95249eebdc9ae86de441023', 'request_id': 'req-6ceaef2e-fafe-4394-bb1e-03d70d08e579'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:56:14.790841 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-6ceaef2e-fafe-4394-bb1e-03d70d08e579 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1490850f95249eebdc9ae86de441023', 'request_id': 'req-6ceaef2e-fafe-4394-bb1e-03d70d08e579'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:56:14.847151 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1282/2571] 10.222.0.81 () {58 vars in 1018 bytes} [Wed Jul 20 22:56:14 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 22:56:14.857615 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc8c336-6e2c-49d9-81e9-24b91ba9e877 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:56:14.882756 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-6ceaef2e-fafe-4394-bb1e-03d70d08e579 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1490850f95249eebdc9ae86de441023', 'request_id': 'req-6ceaef2e-fafe-4394-bb1e-03d70d08e579'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:56:14.890868 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:56:14.898936 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-6ceaef2e-fafe-4394-bb1e-03d70d08e579 admin admin] The token cache is being invalidate because project c1490850f95249eebdc9ae86de441023 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:56:14.898936 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-6ceaef2e-fafe-4394-bb1e-03d70d08e579 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:56:14.898936 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-6ceaef2e-fafe-4394-bb1e-03d70d08e579 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:56:14.898936 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1290/2572] 10.222.0.81 () {58 vars in 1232 bytes} [Wed Jul 20 22:56:14 2022] DELETE /identity/v3/projects/c1490850f95249eebdc9ae86de441023 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 22:56:14.902048 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34474728-2182-4c6a-822f-308926d862db None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:56:14.902415 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34474728-2182-4c6a-822f-308926d862db None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:56:14.902773 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34474728-2182-4c6a-822f-308926d862db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:56:14.911584 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc8c336-6e2c-49d9-81e9-24b91ba9e877 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:56:14.915244 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0d7553-b1d9-4ebe-9ad5-ead0d8896f72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:56:14.915895 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0d7553-b1d9-4ebe-9ad5-ead0d8896f72 service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:56:14.916179 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0d7553-b1d9-4ebe-9ad5-ead0d8896f72 service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:56:14.916459 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0d7553-b1d9-4ebe-9ad5-ead0d8896f72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:56:14.918460 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-2a0d7553-b1d9-4ebe-9ad5-ead0d8896f72 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:56:14.922892 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-34474728-2182-4c6a-822f-308926d862db None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:56:14.924119 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-34474728-2182-4c6a-822f-308926d862db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:56:14.932092 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0d7553-b1d9-4ebe-9ad5-ead0d8896f72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:56:14.933519 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0d7553-b1d9-4ebe-9ad5-ead0d8896f72 service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:56:14.944672 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1283/2573] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:56:14 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 22:56:14.982640 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1291/2574] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:56:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 22:56:14.991884 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83deb3b-288f-440d-8eb5-86f89e8a6d2a None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:56:14.992179 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83deb3b-288f-440d-8eb5-86f89e8a6d2a None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:56:14.992491 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83deb3b-288f-440d-8eb5-86f89e8a6d2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:56:15.010178 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34474728-2182-4c6a-822f-308926d862db None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:56:15.011929 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-34474728-2182-4c6a-822f-308926d862db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:56:15.014130 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-d83deb3b-288f-440d-8eb5-86f89e8a6d2a None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:56:15.015545 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-d83deb3b-288f-440d-8eb5-86f89e8a6d2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:56:15.024164 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1284/2575] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:56:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:56:15.028087 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83deb3b-288f-440d-8eb5-86f89e8a6d2a None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:56:15.038909 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83deb3b-288f-440d-8eb5-86f89e8a6d2a None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:56:15.044228 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af76f797-37ce-47bb-bbfa-20b09904f696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:56:15.045202 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af76f797-37ce-47bb-bbfa-20b09904f696 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:56:15.045583 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af76f797-37ce-47bb-bbfa-20b09904f696 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:56:15.045966 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af76f797-37ce-47bb-bbfa-20b09904f696 admin admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:56:15.047623 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af76f797-37ce-47bb-bbfa-20b09904f696 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:56:15.048050 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af76f797-37ce-47bb-bbfa-20b09904f696 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:56:15.049643 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af76f797-37ce-47bb-bbfa-20b09904f696 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:56:15.066151 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1285/2576] 10.222.0.81 () {58 vars in 1152 bytes} [Wed Jul 20 22:56:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 22:56:15.070426 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af76f797-37ce-47bb-bbfa-20b09904f696 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:56:15.079909 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af76f797-37ce-47bb-bbfa-20b09904f696 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:56:15.084544 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061b69cf-b98a-4d0e-b224-120f5adbdae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:56:15.085202 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061b69cf-b98a-4d0e-b224-120f5adbdae1 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:56:15.085469 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061b69cf-b98a-4d0e-b224-120f5adbdae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:56:15.085749 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061b69cf-b98a-4d0e-b224-120f5adbdae1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:56:15.086432 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061b69cf-b98a-4d0e-b224-120f5adbdae1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:56:15.087705 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34474728-2182-4c6a-822f-308926d862db None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:56:15.088354 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061b69cf-b98a-4d0e-b224-120f5adbdae1 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:56:15.093741 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b438f4-4cd3-40e2-9d58-b2e44b5b8ad1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5500cebdd9d4f8e9aecb45d6c955878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:56:15.098602 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b438f4-4cd3-40e2-9d58-b2e44b5b8ad1 service placement] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:56:15.098602 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b438f4-4cd3-40e2-9d58-b2e44b5b8ad1 service placement] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:56:15.098602 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b438f4-4cd3-40e2-9d58-b2e44b5b8ad1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:56:15.098602 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-00b438f4-4cd3-40e2-9d58-b2e44b5b8ad1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:56:15.108470 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b438f4-4cd3-40e2-9d58-b2e44b5b8ad1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:56:15.110698 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b438f4-4cd3-40e2-9d58-b2e44b5b8ad1 service placement] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:56:15.120771 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1292/2577] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:56:15 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 22:56:15.174179 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1286/2578] 10.222.0.81 () {60 vars in 1152 bytes} [Wed Jul 20 22:56:15 2022] POST /identity/v3/projects => generated 377 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 22:56:15.175436 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b438f4-4cd3-40e2-9d58-b2e44b5b8ad1 service placement] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:56:15.187507 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b438f4-4cd3-40e2-9d58-b2e44b5b8ad1 service placement] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:56:15.251339 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a67ae14-6a69-46c4-ad18-e6f25c0a3b1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:56:15.252032 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a67ae14-6a69-46c4-ad18-e6f25c0a3b1f admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:56:15.252312 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a67ae14-6a69-46c4-ad18-e6f25c0a3b1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:56:15.252593 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a67ae14-6a69-46c4-ad18-e6f25c0a3b1f admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:56:15.253288 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a67ae14-6a69-46c4-ad18-e6f25c0a3b1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:56:15.255412 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a67ae14-6a69-46c4-ad18-e6f25c0a3b1f admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:56:15.342521 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1293/2579] 10.222.0.81 () {60 vars in 1143 bytes} [Wed Jul 20 22:56:15 2022] POST /identity/v3/users => generated 289 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:56:15.344247 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061b69cf-b98a-4d0e-b224-120f5adbdae1 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:56:15.355057 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061b69cf-b98a-4d0e-b224-120f5adbdae1 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:56:15.360480 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94d5a8c-6d8f-4e44-bfa6-1aa214ac983d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:56:15.367142 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94d5a8c-6d8f-4e44-bfa6-1aa214ac983d admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:56:15.367142 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94d5a8c-6d8f-4e44-bfa6-1aa214ac983d admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:56:15.367142 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94d5a8c-6d8f-4e44-bfa6-1aa214ac983d admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:56:15.367142 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94d5a8c-6d8f-4e44-bfa6-1aa214ac983d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:56:15.367142 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94d5a8c-6d8f-4e44-bfa6-1aa214ac983d admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:56:15.375244 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1287/2580] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:56:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 22:56:15.379361 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a67ae14-6a69-46c4-ad18-e6f25c0a3b1f admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:56:15.389756 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a67ae14-6a69-46c4-ad18-e6f25c0a3b1f admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:56:15.394385 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97b3797-e7c1-46ed-b3c6-105f765a4eb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:56:15.395089 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97b3797-e7c1-46ed-b3c6-105f765a4eb8 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:56:15.395383 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97b3797-e7c1-46ed-b3c6-105f765a4eb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:56:15.395719 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97b3797-e7c1-46ed-b3c6-105f765a4eb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:56:15.396446 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97b3797-e7c1-46ed-b3c6-105f765a4eb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:56:15.398035 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97b3797-e7c1-46ed-b3c6-105f765a4eb8 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:56:15.412068 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1294/2581] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:56:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 22:56:15.413101 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94d5a8c-6d8f-4e44-bfa6-1aa214ac983d admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:56:15.424233 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94d5a8c-6d8f-4e44-bfa6-1aa214ac983d admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:56:15.429541 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5769a800-fd06-44a8-9f69-bb1211e3106e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:56:15.430559 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5769a800-fd06-44a8-9f69-bb1211e3106e admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:56:15.430965 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5769a800-fd06-44a8-9f69-bb1211e3106e admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:56:15.431387 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5769a800-fd06-44a8-9f69-bb1211e3106e admin admin] PATH_INFO: `/v3/projects/bb55751359e34bdca736759fa4dc4b7f/users/27e9447f68f74e30b8400879be541d19/roles/559942c4de1744169f8b5e0404ecde6b` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:56:15.446670 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5769a800-fd06-44a8-9f69-bb1211e3106e admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb55751359e34bdca736759fa4dc4b7f, user_id=27e9447f68f74e30b8400879be541d19, role_id=559942c4de1744169f8b5e0404ecde6b)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:56:15.449499 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5769a800-fd06-44a8-9f69-bb1211e3106e admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:56:15.523235 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1288/2582] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:56:15 2022] PUT /identity/v3/projects/bb55751359e34bdca736759fa4dc4b7f/users/27e9447f68f74e30b8400879be541d19/roles/559942c4de1744169f8b5e0404ecde6b => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 22:56:15.534003 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0564d4f8-ea45-4b41-bef6-4c99d70127f1 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:56:15.534470 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0564d4f8-ea45-4b41-bef6-4c99d70127f1 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:56:15.534597 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0564d4f8-ea45-4b41-bef6-4c99d70127f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:56:15.553964 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:56:15.563053 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-0564d4f8-ea45-4b41-bef6-4c99d70127f1 None None] MFA Rules not processed for user `27e9447f68f74e30b8400879be541d19`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:56:15.564203 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-0564d4f8-ea45-4b41-bef6-4c99d70127f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:56:15.631579 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1295/2583] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:56:15 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:56:15.643618 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5769a800-fd06-44a8-9f69-bb1211e3106e admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:56:15.645325 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-5769a800-fd06-44a8-9f69-bb1211e3106e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:56:15.677064 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:56:15.704656 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5769a800-fd06-44a8-9f69-bb1211e3106e admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:56:15.709044 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420340d7-83f3-42fb-8381-9820bde0a917 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecf4db289fba4e4a806198c62d31f7c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:56:15.710103 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420340d7-83f3-42fb-8381-9820bde0a917 service cinder] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:56:15.710498 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420340d7-83f3-42fb-8381-9820bde0a917 service cinder] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:56:15.710818 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420340d7-83f3-42fb-8381-9820bde0a917 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:56:15.726754 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420340d7-83f3-42fb-8381-9820bde0a917 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:56:15.729650 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420340d7-83f3-42fb-8381-9820bde0a917 service cinder] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:56:15.739417 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1289/2584] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:56:15 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:56:25.642064 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=28 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:56:25.751118 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=14 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:56:30.185566 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4925b039-3f68-446e-b7a1-13c1c55c44a9 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:56:30.186051 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4925b039-3f68-446e-b7a1-13c1c55c44a9 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:56:30.186150 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4925b039-3f68-446e-b7a1-13c1c55c44a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:56:30.208941 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-4925b039-3f68-446e-b7a1-13c1c55c44a9 None None] MFA Rules not processed for user `90915e1264324f16bc7bae7e46e0c97b`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:56:30.210198 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-4925b039-3f68-446e-b7a1-13c1c55c44a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:56:30.268047 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1296/2585] 10.222.0.81 () {58 vars in 967 bytes} [Wed Jul 20 22:56:30 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 22:56:30.280507 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420340d7-83f3-42fb-8381-9820bde0a917 service cinder] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:56:30.281891 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-420340d7-83f3-42fb-8381-9820bde0a917 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:56:30.350382 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420340d7-83f3-42fb-8381-9820bde0a917 service cinder] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:56:30.355431 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf9b595-e3c0-4377-a383-9fef0f856964 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70e97436a08543a180225561ef8bc817', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:56:30.356081 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf9b595-e3c0-4377-a383-9fef0f856964 service swift] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:56:30.356369 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf9b595-e3c0-4377-a383-9fef0f856964 service swift] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:56:30.356655 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf9b595-e3c0-4377-a383-9fef0f856964 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:56:30.364784 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf9b595-e3c0-4377-a383-9fef0f856964 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:56:30.366359 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf9b595-e3c0-4377-a383-9fef0f856964 service swift] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:56:30.374794 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1290/2586] 10.222.0.81 () {60 vars in 1384 bytes} [Wed Jul 20 22:56:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 22:56:40.259693 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=25 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:56:40.371093 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=32 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:56:44.810713 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4925b039-3f68-446e-b7a1-13c1c55c44a9 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:56:44.874662 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4925b039-3f68-446e-b7a1-13c1c55c44a9 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:56:44.880047 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaa13fb-d4e7-403d-9867-a0025cfd0c13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:56:44.880950 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaa13fb-d4e7-403d-9867-a0025cfd0c13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:56:44.881242 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaa13fb-d4e7-403d-9867-a0025cfd0c13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:56:44.881528 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaa13fb-d4e7-403d-9867-a0025cfd0c13 admin admin] PATH_INFO: `/v3/users/27e9447f68f74e30b8400879be541d19` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:56:44.882817 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aaa13fb-d4e7-403d-9867-a0025cfd0c13 admin admin] RBAC: Authorizing `identity:delete_user(user_id=27e9447f68f74e30b8400879be541d19)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:56:44.884722 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aaa13fb-d4e7-403d-9867-a0025cfd0c13 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:56:45.096043 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-2aaa13fb-d4e7-403d-9867-a0025cfd0c13 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27e9447f68f74e30b8400879be541d19'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:56:45.096695 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-2aaa13fb-d4e7-403d-9867-a0025cfd0c13 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27e9447f68f74e30b8400879be541d19'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:56:45.097302 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-2aaa13fb-d4e7-403d-9867-a0025cfd0c13 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27e9447f68f74e30b8400879be541d19'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:56:45.104544 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-2aaa13fb-d4e7-403d-9867-a0025cfd0c13 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27e9447f68f74e30b8400879be541d19'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:56:45.174720 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-2aaa13fb-d4e7-403d-9867-a0025cfd0c13 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27e9447f68f74e30b8400879be541d19'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:56:45.202374 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1297/2587] 10.222.0.81 () {58 vars in 1223 bytes} [Wed Jul 20 22:56:44 2022] DELETE /identity/v3/users/27e9447f68f74e30b8400879be541d19 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:56:45.207190 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691bd933-84e3-4e2b-ac95-6382691d4a4c None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:56:45.207681 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691bd933-84e3-4e2b-ac95-6382691d4a4c None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:56:45.207681 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691bd933-84e3-4e2b-ac95-6382691d4a4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:56:45.226969 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-691bd933-84e3-4e2b-ac95-6382691d4a4c None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:56:45.228241 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-691bd933-84e3-4e2b-ac95-6382691d4a4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:56:45.300884 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1291/2588] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:56:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 22:56:45.312336 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaa13fb-d4e7-403d-9867-a0025cfd0c13 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:56:45.314124 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-2aaa13fb-d4e7-403d-9867-a0025cfd0c13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:56:45.405974 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaa13fb-d4e7-403d-9867-a0025cfd0c13 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:56:45.412101 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c63049f-27cb-4efc-9b0c-7a54fe435a0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:56:45.412838 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c63049f-27cb-4efc-9b0c-7a54fe435a0d service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:56:45.413201 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c63049f-27cb-4efc-9b0c-7a54fe435a0d service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:56:45.413564 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c63049f-27cb-4efc-9b0c-7a54fe435a0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:56:45.425285 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c63049f-27cb-4efc-9b0c-7a54fe435a0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:56:45.426738 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c63049f-27cb-4efc-9b0c-7a54fe435a0d service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:56:45.437774 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1298/2589] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:56:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:56:45.872004 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691bd933-84e3-4e2b-ac95-6382691d4a4c None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:56:45.873561 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-691bd933-84e3-4e2b-ac95-6382691d4a4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:56:45.886171 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691bd933-84e3-4e2b-ac95-6382691d4a4c None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:56:45.891020 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1275981-d40b-48e5-abfa-eb8f3639a4a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:56:45.892404 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1275981-d40b-48e5-abfa-eb8f3639a4a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:56:45.892694 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1275981-d40b-48e5-abfa-eb8f3639a4a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:56:45.893063 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1275981-d40b-48e5-abfa-eb8f3639a4a4 admin admin] PATH_INFO: `/v3/projects/bb55751359e34bdca736759fa4dc4b7f` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:56:45.894181 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1275981-d40b-48e5-abfa-eb8f3639a4a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb55751359e34bdca736759fa4dc4b7f)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:56:45.896207 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1275981-d40b-48e5-abfa-eb8f3639a4a4 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:56:46.015479 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-a1275981-d40b-48e5-abfa-eb8f3639a4a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb55751359e34bdca736759fa4dc4b7f', 'request_id': 'req-a1275981-d40b-48e5-abfa-eb8f3639a4a4'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:56:46.083156 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-a1275981-d40b-48e5-abfa-eb8f3639a4a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb55751359e34bdca736759fa4dc4b7f', 'request_id': 'req-a1275981-d40b-48e5-abfa-eb8f3639a4a4'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:56:46.089276 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-a1275981-d40b-48e5-abfa-eb8f3639a4a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb55751359e34bdca736759fa4dc4b7f', 'request_id': 'req-a1275981-d40b-48e5-abfa-eb8f3639a4a4'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:56:46.100708 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-a1275981-d40b-48e5-abfa-eb8f3639a4a4 admin admin] The token cache is being invalidate because project bb55751359e34bdca736759fa4dc4b7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:56:46.104649 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-a1275981-d40b-48e5-abfa-eb8f3639a4a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:56:46.104649 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-a1275981-d40b-48e5-abfa-eb8f3639a4a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:56:46.104649 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1292/2590] 10.222.0.81 () {58 vars in 1232 bytes} [Wed Jul 20 22:56:45 2022] DELETE /identity/v3/projects/bb55751359e34bdca736759fa4dc4b7f => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:56:46.108386 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5503370-758e-4bd9-a766-975edf2f410b None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:56:46.108854 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5503370-758e-4bd9-a766-975edf2f410b None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:56:46.108949 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5503370-758e-4bd9-a766-975edf2f410b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:56:46.126879 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-c5503370-758e-4bd9-a766-975edf2f410b None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:56:46.128134 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-c5503370-758e-4bd9-a766-975edf2f410b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:56:46.145743 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:56:46.187842 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1299/2591] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:56:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:56:46.251602 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f8f7ec-2baf-4cc4-8956-b8724f9f9ffc None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:56:46.252171 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f8f7ec-2baf-4cc4-8956-b8724f9f9ffc None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:56:46.252171 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f8f7ec-2baf-4cc4-8956-b8724f9f9ffc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:56:46.271380 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-d8f8f7ec-2baf-4cc4-8956-b8724f9f9ffc None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:56:46.272572 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-d8f8f7ec-2baf-4cc4-8956-b8724f9f9ffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:56:46.280312 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1293/2592] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:56:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:56:46.284438 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5503370-758e-4bd9-a766-975edf2f410b None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:56:46.294134 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5503370-758e-4bd9-a766-975edf2f410b None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:56:46.298682 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149f2e7b-16f7-41fb-b723-798309cbd4e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:56:46.299375 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149f2e7b-16f7-41fb-b723-798309cbd4e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:56:46.299665 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149f2e7b-16f7-41fb-b723-798309cbd4e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:56:46.299945 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149f2e7b-16f7-41fb-b723-798309cbd4e4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:56:46.300665 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149f2e7b-16f7-41fb-b723-798309cbd4e4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:56:46.300978 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149f2e7b-16f7-41fb-b723-798309cbd4e4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:56:46.302577 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149f2e7b-16f7-41fb-b723-798309cbd4e4 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:56:46.314518 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1300/2593] 10.222.0.81 () {58 vars in 1152 bytes} [Wed Jul 20 22:56:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:56:46.318680 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f8f7ec-2baf-4cc4-8956-b8724f9f9ffc None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:56:46.327910 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f8f7ec-2baf-4cc4-8956-b8724f9f9ffc None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:56:46.333104 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934d8cb0-2c1d-44fd-8481-44ce3ce5c182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:56:46.333741 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934d8cb0-2c1d-44fd-8481-44ce3ce5c182 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:56:46.334023 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934d8cb0-2c1d-44fd-8481-44ce3ce5c182 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:56:46.334299 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934d8cb0-2c1d-44fd-8481-44ce3ce5c182 admin admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:56:46.334982 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934d8cb0-2c1d-44fd-8481-44ce3ce5c182 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:56:46.336924 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934d8cb0-2c1d-44fd-8481-44ce3ce5c182 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:56:46.422168 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1294/2594] 10.222.0.81 () {60 vars in 1152 bytes} [Wed Jul 20 22:56:46 2022] POST /identity/v3/projects => generated 391 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:56:46.427024 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149f2e7b-16f7-41fb-b723-798309cbd4e4 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:56:46.437754 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149f2e7b-16f7-41fb-b723-798309cbd4e4 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:56:46.498752 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bee902-19b4-49b6-8a43-a08ea0dec776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:56:46.499554 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bee902-19b4-49b6-8a43-a08ea0dec776 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:56:46.499845 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bee902-19b4-49b6-8a43-a08ea0dec776 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:56:46.500149 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bee902-19b4-49b6-8a43-a08ea0dec776 admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:56:46.500846 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bee902-19b4-49b6-8a43-a08ea0dec776 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:56:46.503452 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bee902-19b4-49b6-8a43-a08ea0dec776 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:56:46.586458 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1301/2595] 10.222.0.81 () {60 vars in 1143 bytes} [Wed Jul 20 22:56:46 2022] POST /identity/v3/users => generated 296 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:56:46.590364 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934d8cb0-2c1d-44fd-8481-44ce3ce5c182 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:56:46.600771 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934d8cb0-2c1d-44fd-8481-44ce3ce5c182 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:56:46.605419 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7b3157-57dd-42a6-bafd-67cfbe6a7430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:56:46.606076 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7b3157-57dd-42a6-bafd-67cfbe6a7430 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:56:46.606364 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7b3157-57dd-42a6-bafd-67cfbe6a7430 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:56:46.606754 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7b3157-57dd-42a6-bafd-67cfbe6a7430 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:56:46.607444 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7b3157-57dd-42a6-bafd-67cfbe6a7430 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:56:46.609031 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7b3157-57dd-42a6-bafd-67cfbe6a7430 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:56:46.620883 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1295/2596] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:56:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:56:46.625092 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bee902-19b4-49b6-8a43-a08ea0dec776 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:56:46.635857 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bee902-19b4-49b6-8a43-a08ea0dec776 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:56:46.640762 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1e131f-e8b3-4e69-b8d3-33a60be4fb14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:56:46.641432 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1e131f-e8b3-4e69-b8d3-33a60be4fb14 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:56:46.641716 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1e131f-e8b3-4e69-b8d3-33a60be4fb14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:56:46.641997 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1e131f-e8b3-4e69-b8d3-33a60be4fb14 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:56:46.642725 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1e131f-e8b3-4e69-b8d3-33a60be4fb14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:56:46.644340 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1e131f-e8b3-4e69-b8d3-33a60be4fb14 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:56:46.654164 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1302/2597] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:56:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 22:56:46.658109 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7b3157-57dd-42a6-bafd-67cfbe6a7430 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:56:46.667863 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7b3157-57dd-42a6-bafd-67cfbe6a7430 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:56:46.672416 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67adae2-50b4-40e8-abee-8cfdf02afe0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:56:46.673826 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67adae2-50b4-40e8-abee-8cfdf02afe0e admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:56:46.674106 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67adae2-50b4-40e8-abee-8cfdf02afe0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:56:46.674379 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67adae2-50b4-40e8-abee-8cfdf02afe0e admin admin] PATH_INFO: `/v3/projects/fe3edc44b7ee47139e03ec1dc3787da8/users/f8cfed4bb67e4bc78072ec3554879771/roles/559942c4de1744169f8b5e0404ecde6b` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:56:46.688370 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67adae2-50b4-40e8-abee-8cfdf02afe0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe3edc44b7ee47139e03ec1dc3787da8, user_id=f8cfed4bb67e4bc78072ec3554879771, role_id=559942c4de1744169f8b5e0404ecde6b)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:56:46.691578 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67adae2-50b4-40e8-abee-8cfdf02afe0e admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:56:46.785971 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1296/2598] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:56:46 2022] PUT /identity/v3/projects/fe3edc44b7ee47139e03ec1dc3787da8/users/f8cfed4bb67e4bc78072ec3554879771/roles/559942c4de1744169f8b5e0404ecde6b => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:56:46.796104 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae9b709-031e-4d57-a9cf-fce9a2643355 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:56:46.796401 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae9b709-031e-4d57-a9cf-fce9a2643355 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:56:46.796685 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae9b709-031e-4d57-a9cf-fce9a2643355 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:56:46.822724 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-dae9b709-031e-4d57-a9cf-fce9a2643355 None None] MFA Rules not processed for user `f8cfed4bb67e4bc78072ec3554879771`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:56:46.823985 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-dae9b709-031e-4d57-a9cf-fce9a2643355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:56:46.886010 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1303/2599] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:56:46 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:56:46.897157 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67adae2-50b4-40e8-abee-8cfdf02afe0e admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:56:46.898829 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-d67adae2-50b4-40e8-abee-8cfdf02afe0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:56:46.920600 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:56:46.955714 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67adae2-50b4-40e8-abee-8cfdf02afe0e admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:56:46.960396 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cb9b09-ae43-4c66-b5db-e0b22898a39c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecf4db289fba4e4a806198c62d31f7c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:56:46.961072 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cb9b09-ae43-4c66-b5db-e0b22898a39c service cinder] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:56:46.961387 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cb9b09-ae43-4c66-b5db-e0b22898a39c service cinder] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:56:46.961742 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cb9b09-ae43-4c66-b5db-e0b22898a39c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:56:46.970059 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cb9b09-ae43-4c66-b5db-e0b22898a39c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:56:46.971553 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cb9b09-ae43-4c66-b5db-e0b22898a39c service cinder] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:56:46.980361 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1297/2600] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:56:46 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:56:48.570595 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae9b709-031e-4d57-a9cf-fce9a2643355 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:56:48.572721 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-dae9b709-031e-4d57-a9cf-fce9a2643355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:56:48.593367 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:56:48.649929 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae9b709-031e-4d57-a9cf-fce9a2643355 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:56:48.661197 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105c1cea-ca0e-4216-ba78-cca3fb3a52fa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7829324db2574d38b18c749d0559177f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:56:48.661963 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105c1cea-ca0e-4216-ba78-cca3fb3a52fa service nova] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:56:48.662319 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105c1cea-ca0e-4216-ba78-cca3fb3a52fa service nova] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:56:48.662668 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105c1cea-ca0e-4216-ba78-cca3fb3a52fa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:56:48.665096 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-105c1cea-ca0e-4216-ba78-cca3fb3a52fa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:56:48.681318 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105c1cea-ca0e-4216-ba78-cca3fb3a52fa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:56:48.692738 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105c1cea-ca0e-4216-ba78-cca3fb3a52fa service nova] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:56:48.718396 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1304/2601] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:56:48 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:56:57.014366 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=17 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:56:58.763148 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=26 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:57:00.166512 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cb9b09-ae43-4c66-b5db-e0b22898a39c service cinder] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:57:00.168418 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-28cb9b09-ae43-4c66-b5db-e0b22898a39c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:57:00.238835 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cb9b09-ae43-4c66-b5db-e0b22898a39c service cinder] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:57:00.242807 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596caf02-df1f-4af9-b888-1c627a7fb6e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:57:00.243682 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596caf02-df1f-4af9-b888-1c627a7fb6e3 service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:57:00.244085 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596caf02-df1f-4af9-b888-1c627a7fb6e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:57:00.244466 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596caf02-df1f-4af9-b888-1c627a7fb6e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:57:00.246422 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-596caf02-df1f-4af9-b888-1c627a7fb6e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:57:00.318344 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596caf02-df1f-4af9-b888-1c627a7fb6e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:57:00.319756 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596caf02-df1f-4af9-b888-1c627a7fb6e3 service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:57:00.330425 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1298/2602] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:57:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:57:03.997946 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105c1cea-ca0e-4216-ba78-cca3fb3a52fa service nova] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:57:03.997946 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-105c1cea-ca0e-4216-ba78-cca3fb3a52fa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:57:04.002178 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105c1cea-ca0e-4216-ba78-cca3fb3a52fa service nova] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:57:04.006735 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94e8ddd-d143-4eb9-9f2e-25ea69a2ea45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:57:04.007648 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94e8ddd-d143-4eb9-9f2e-25ea69a2ea45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:57:04.007939 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94e8ddd-d143-4eb9-9f2e-25ea69a2ea45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:57:04.008213 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94e8ddd-d143-4eb9-9f2e-25ea69a2ea45 admin admin] PATH_INFO: `/v3/users/3e072e59bf3e46aaa062ec85b0e2c91a` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:57:04.024224 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94e8ddd-d143-4eb9-9f2e-25ea69a2ea45 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e072e59bf3e46aaa062ec85b0e2c91a)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:57:04.026209 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94e8ddd-d143-4eb9-9f2e-25ea69a2ea45 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:57:04.197880 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-e94e8ddd-d143-4eb9-9f2e-25ea69a2ea45 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e072e59bf3e46aaa062ec85b0e2c91a'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:57:04.198574 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-e94e8ddd-d143-4eb9-9f2e-25ea69a2ea45 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e072e59bf3e46aaa062ec85b0e2c91a'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:57:04.199276 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-e94e8ddd-d143-4eb9-9f2e-25ea69a2ea45 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e072e59bf3e46aaa062ec85b0e2c91a'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:57:04.207468 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-e94e8ddd-d143-4eb9-9f2e-25ea69a2ea45 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e072e59bf3e46aaa062ec85b0e2c91a'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:57:04.299634 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-e94e8ddd-d143-4eb9-9f2e-25ea69a2ea45 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e072e59bf3e46aaa062ec85b0e2c91a'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:57:04.317888 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1305/2603] 10.222.0.81 () {58 vars in 1223 bytes} [Wed Jul 20 22:57:03 2022] DELETE /identity/v3/users/3e072e59bf3e46aaa062ec85b0e2c91a => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 22:57:04.599043 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596caf02-df1f-4af9-b888-1c627a7fb6e3 service neutron] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:57:04.610158 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596caf02-df1f-4af9-b888-1c627a7fb6e3 service neutron] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:57:04.669198 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de1397b-5e99-4128-a33b-17001de793f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:57:04.670081 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de1397b-5e99-4128-a33b-17001de793f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:57:04.670372 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de1397b-5e99-4128-a33b-17001de793f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:57:04.670660 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de1397b-5e99-4128-a33b-17001de793f2 admin admin] PATH_INFO: `/v3/projects/f733781536584f80acfc325bf8444536` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:57:04.681444 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de1397b-5e99-4128-a33b-17001de793f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f733781536584f80acfc325bf8444536)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:57:04.683397 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de1397b-5e99-4128-a33b-17001de793f2 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:57:04.788571 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:57:04.792218 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-6de1397b-5e99-4128-a33b-17001de793f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f733781536584f80acfc325bf8444536', 'request_id': 'req-6de1397b-5e99-4128-a33b-17001de793f2'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:57:04.853980 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-6de1397b-5e99-4128-a33b-17001de793f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f733781536584f80acfc325bf8444536', 'request_id': 'req-6de1397b-5e99-4128-a33b-17001de793f2'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:57:04.859779 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-6de1397b-5e99-4128-a33b-17001de793f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f733781536584f80acfc325bf8444536', 'request_id': 'req-6de1397b-5e99-4128-a33b-17001de793f2'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:57:04.871137 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-6de1397b-5e99-4128-a33b-17001de793f2 admin admin] The token cache is being invalidate because project f733781536584f80acfc325bf8444536 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:57:04.871427 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-6de1397b-5e99-4128-a33b-17001de793f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:57:04.872109 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-6de1397b-5e99-4128-a33b-17001de793f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:57:04.873615 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1299/2604] 10.222.0.81 () {58 vars in 1232 bytes} [Wed Jul 20 22:57:04 2022] DELETE /identity/v3/projects/f733781536584f80acfc325bf8444536 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:57:04.879091 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c229b73-83b5-45d1-a1d2-9fb6fa9b823c None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:57:04.879448 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c229b73-83b5-45d1-a1d2-9fb6fa9b823c None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:57:04.879730 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c229b73-83b5-45d1-a1d2-9fb6fa9b823c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:57:04.897420 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-4c229b73-83b5-45d1-a1d2-9fb6fa9b823c None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:57:04.898620 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-4c229b73-83b5-45d1-a1d2-9fb6fa9b823c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:57:04.962295 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1306/2605] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:57:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:57:04.970931 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749e1e8d-1e41-4573-a533-e8d6ccfad25f None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:57:04.971251 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749e1e8d-1e41-4573-a533-e8d6ccfad25f None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:57:04.971536 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749e1e8d-1e41-4573-a533-e8d6ccfad25f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:57:04.989059 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-749e1e8d-1e41-4573-a533-e8d6ccfad25f None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:57:04.990247 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-749e1e8d-1e41-4573-a533-e8d6ccfad25f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:57:04.998075 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1300/2606] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:57:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:57:05.002586 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c229b73-83b5-45d1-a1d2-9fb6fa9b823c None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:57:05.012406 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c229b73-83b5-45d1-a1d2-9fb6fa9b823c None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:57:05.017604 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdcd435-9179-4966-92ab-aeb12990f918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:57:05.018258 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdcd435-9179-4966-92ab-aeb12990f918 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:57:05.018539 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdcd435-9179-4966-92ab-aeb12990f918 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:57:05.018814 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdcd435-9179-4966-92ab-aeb12990f918 admin admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:57:05.019568 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdcd435-9179-4966-92ab-aeb12990f918 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:57:05.019880 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdcd435-9179-4966-92ab-aeb12990f918 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:57:05.021486 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdcd435-9179-4966-92ab-aeb12990f918 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:57:05.032956 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1307/2607] 10.222.0.81 () {58 vars in 1152 bytes} [Wed Jul 20 22:57:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:57:05.037041 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749e1e8d-1e41-4573-a533-e8d6ccfad25f None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:57:05.046203 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749e1e8d-1e41-4573-a533-e8d6ccfad25f None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:57:05.050848 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2553d3cd-46c3-4545-9909-064bcf6572f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:57:05.051525 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2553d3cd-46c3-4545-9909-064bcf6572f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:57:05.051810 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2553d3cd-46c3-4545-9909-064bcf6572f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:57:05.052086 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2553d3cd-46c3-4545-9909-064bcf6572f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:57:05.052774 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2553d3cd-46c3-4545-9909-064bcf6572f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:57:05.054792 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2553d3cd-46c3-4545-9909-064bcf6572f5 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:57:05.141248 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1301/2608] 10.222.0.81 () {60 vars in 1152 bytes} [Wed Jul 20 22:57:05 2022] POST /identity/v3/projects => generated 367 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:57:05.145896 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdcd435-9179-4966-92ab-aeb12990f918 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:57:05.155803 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdcd435-9179-4966-92ab-aeb12990f918 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:57:05.201225 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:57:05.213675 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82dac3a-8334-4e7b-bb49-1bb4768781c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:57:05.214359 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82dac3a-8334-4e7b-bb49-1bb4768781c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:57:05.214648 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82dac3a-8334-4e7b-bb49-1bb4768781c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:57:05.214935 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82dac3a-8334-4e7b-bb49-1bb4768781c9 admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:57:05.215658 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82dac3a-8334-4e7b-bb49-1bb4768781c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:57:05.217589 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82dac3a-8334-4e7b-bb49-1bb4768781c9 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:57:05.300519 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1308/2609] 10.222.0.81 () {60 vars in 1143 bytes} [Wed Jul 20 22:57:05 2022] POST /identity/v3/users => generated 284 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:57:05.304492 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2553d3cd-46c3-4545-9909-064bcf6572f5 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:57:05.315166 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2553d3cd-46c3-4545-9909-064bcf6572f5 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:57:05.319877 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21e86c2-d000-4143-92f0-25ce082afe7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:57:05.320542 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21e86c2-d000-4143-92f0-25ce082afe7c admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:57:05.320826 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21e86c2-d000-4143-92f0-25ce082afe7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:57:05.321157 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21e86c2-d000-4143-92f0-25ce082afe7c admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:57:05.321887 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21e86c2-d000-4143-92f0-25ce082afe7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:57:05.323516 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21e86c2-d000-4143-92f0-25ce082afe7c admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:57:05.334686 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1302/2610] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:57:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:57:05.339034 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82dac3a-8334-4e7b-bb49-1bb4768781c9 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:57:05.350464 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82dac3a-8334-4e7b-bb49-1bb4768781c9 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:57:05.355234 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841b1c50-2a1e-423f-a844-dd5a579b160b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:57:05.355935 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841b1c50-2a1e-423f-a844-dd5a579b160b admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:57:05.356236 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841b1c50-2a1e-423f-a844-dd5a579b160b admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:57:05.356501 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841b1c50-2a1e-423f-a844-dd5a579b160b admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:57:05.357263 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841b1c50-2a1e-423f-a844-dd5a579b160b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:57:05.358937 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841b1c50-2a1e-423f-a844-dd5a579b160b admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:57:05.373001 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1309/2611] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:57:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:57:05.374353 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21e86c2-d000-4143-92f0-25ce082afe7c admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:57:05.384759 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21e86c2-d000-4143-92f0-25ce082afe7c admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:57:05.389985 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f6ebee-09ae-4933-bf26-92eb637d208f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:57:05.390884 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f6ebee-09ae-4933-bf26-92eb637d208f admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:57:05.391215 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f6ebee-09ae-4933-bf26-92eb637d208f admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:57:05.391586 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f6ebee-09ae-4933-bf26-92eb637d208f admin admin] PATH_INFO: `/v3/projects/1cbc2d06b7fb4a9a80f0fa9e19aaf21c/users/5236f2caddd34a90a7853864de2bbac7/roles/559942c4de1744169f8b5e0404ecde6b` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:57:05.410855 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f6ebee-09ae-4933-bf26-92eb637d208f admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cbc2d06b7fb4a9a80f0fa9e19aaf21c, user_id=5236f2caddd34a90a7853864de2bbac7, role_id=559942c4de1744169f8b5e0404ecde6b)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:57:05.414006 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f6ebee-09ae-4933-bf26-92eb637d208f admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:57:05.495391 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1303/2612] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:57:05 2022] PUT /identity/v3/projects/1cbc2d06b7fb4a9a80f0fa9e19aaf21c/users/5236f2caddd34a90a7853864de2bbac7/roles/559942c4de1744169f8b5e0404ecde6b => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:57:05.506678 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c4aff7-8beb-4ce4-a11c-2f9df6ad2272 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:57:05.506969 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c4aff7-8beb-4ce4-a11c-2f9df6ad2272 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:57:05.507283 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c4aff7-8beb-4ce4-a11c-2f9df6ad2272 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:57:05.534386 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-b1c4aff7-8beb-4ce4-a11c-2f9df6ad2272 None None] MFA Rules not processed for user `5236f2caddd34a90a7853864de2bbac7`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:57:05.535672 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-b1c4aff7-8beb-4ce4-a11c-2f9df6ad2272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:57:05.605824 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1310/2613] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:57:05 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:57:05.612127 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f6ebee-09ae-4933-bf26-92eb637d208f admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:57:05.683247 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f6ebee-09ae-4933-bf26-92eb637d208f admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:57:05.687822 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac7d67f-0752-44ee-bb71-bf233e74d363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:57:05.688505 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac7d67f-0752-44ee-bb71-bf233e74d363 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:57:05.688797 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac7d67f-0752-44ee-bb71-bf233e74d363 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:57:05.689076 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac7d67f-0752-44ee-bb71-bf233e74d363 admin admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:57:05.689755 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac7d67f-0752-44ee-bb71-bf233e74d363 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:57:05.691678 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac7d67f-0752-44ee-bb71-bf233e74d363 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:57:05.808697 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1304/2614] 10.222.0.81 () {60 vars in 1152 bytes} [Wed Jul 20 22:57:05 2022] POST /identity/v3/projects => generated 367 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:57:05.809863 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c4aff7-8beb-4ce4-a11c-2f9df6ad2272 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:57:05.820793 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c4aff7-8beb-4ce4-a11c-2f9df6ad2272 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:57:05.889771 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4301ffe2-e523-45b9-af02-152e0d80cca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:57:05.890459 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4301ffe2-e523-45b9-af02-152e0d80cca8 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:57:05.890751 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4301ffe2-e523-45b9-af02-152e0d80cca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:57:05.891059 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4301ffe2-e523-45b9-af02-152e0d80cca8 admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:57:05.892289 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4301ffe2-e523-45b9-af02-152e0d80cca8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:57:05.894188 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4301ffe2-e523-45b9-af02-152e0d80cca8 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:57:06.006362 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1311/2615] 10.222.0.81 () {60 vars in 1143 bytes} [Wed Jul 20 22:57:05 2022] POST /identity/v3/users => generated 290 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:57:06.008300 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac7d67f-0752-44ee-bb71-bf233e74d363 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:57:06.022995 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac7d67f-0752-44ee-bb71-bf233e74d363 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:57:06.027999 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42049c2b-2732-496f-9544-3ab17abe1ca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:57:06.028747 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42049c2b-2732-496f-9544-3ab17abe1ca7 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:57:06.028944 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42049c2b-2732-496f-9544-3ab17abe1ca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:57:06.029222 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42049c2b-2732-496f-9544-3ab17abe1ca7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:57:06.029939 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42049c2b-2732-496f-9544-3ab17abe1ca7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:57:06.031615 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42049c2b-2732-496f-9544-3ab17abe1ca7 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:57:06.043960 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1305/2616] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:57:06 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:57:06.048391 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4301ffe2-e523-45b9-af02-152e0d80cca8 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:57:06.058702 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4301ffe2-e523-45b9-af02-152e0d80cca8 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:57:06.063430 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1b6a1c-c0df-4dd9-a249-e9238e37e69b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:57:06.064369 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1b6a1c-c0df-4dd9-a249-e9238e37e69b admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:57:06.064656 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1b6a1c-c0df-4dd9-a249-e9238e37e69b admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:57:06.064937 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1b6a1c-c0df-4dd9-a249-e9238e37e69b admin admin] PATH_INFO: `/v3/projects/b56b868f8933464db7abb77435e5d095/users/d4a0edb5f7e047bd9e669ab9ce91fac7/roles/d89b6c92d35a401a8517f1d47f6be843` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:57:06.081575 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1b6a1c-c0df-4dd9-a249-e9238e37e69b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b56b868f8933464db7abb77435e5d095, user_id=d4a0edb5f7e047bd9e669ab9ce91fac7, role_id=d89b6c92d35a401a8517f1d47f6be843)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:57:06.084424 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1b6a1c-c0df-4dd9-a249-e9238e37e69b admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:57:06.152351 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1312/2617] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:57:06 2022] PUT /identity/v3/projects/b56b868f8933464db7abb77435e5d095/users/d4a0edb5f7e047bd9e669ab9ce91fac7/roles/d89b6c92d35a401a8517f1d47f6be843 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 22:57:06.156588 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42049c2b-2732-496f-9544-3ab17abe1ca7 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:57:06.167254 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42049c2b-2732-496f-9544-3ab17abe1ca7 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:57:06.209293 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:57:06.224884 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c623204c-ba44-4721-966c-2092baa17684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:57:06.225558 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c623204c-ba44-4721-966c-2092baa17684 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:57:06.225905 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c623204c-ba44-4721-966c-2092baa17684 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:57:06.226197 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c623204c-ba44-4721-966c-2092baa17684 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:57:06.226923 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c623204c-ba44-4721-966c-2092baa17684 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:57:06.228584 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c623204c-ba44-4721-966c-2092baa17684 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:57:06.238959 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1306/2618] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:57:06 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 22:57:06.243292 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1b6a1c-c0df-4dd9-a249-e9238e37e69b admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:57:06.254289 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1b6a1c-c0df-4dd9-a249-e9238e37e69b admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:57:06.258916 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68f089d-4a96-4a8a-ae5e-fb5d4159c86d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:57:06.259901 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68f089d-4a96-4a8a-ae5e-fb5d4159c86d admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:57:06.260446 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68f089d-4a96-4a8a-ae5e-fb5d4159c86d admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:57:06.260752 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68f089d-4a96-4a8a-ae5e-fb5d4159c86d admin admin] PATH_INFO: `/v3/projects/b56b868f8933464db7abb77435e5d095/users/d4a0edb5f7e047bd9e669ab9ce91fac7/roles/d89b6c92d35a401a8517f1d47f6be843` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:57:06.262692 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68f089d-4a96-4a8a-ae5e-fb5d4159c86d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b56b868f8933464db7abb77435e5d095, user_id=d4a0edb5f7e047bd9e669ab9ce91fac7, role_id=d89b6c92d35a401a8517f1d47f6be843)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:57:06.265498 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68f089d-4a96-4a8a-ae5e-fb5d4159c86d admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:57:06.276501 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1313/2619] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:57:06 2022] PUT /identity/v3/projects/b56b868f8933464db7abb77435e5d095/users/d4a0edb5f7e047bd9e669ab9ce91fac7/roles/d89b6c92d35a401a8517f1d47f6be843 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:57:06.286632 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fdebf0-48fd-4621-baf4-2d4ec95031b6 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:57:06.287040 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fdebf0-48fd-4621-baf4-2d4ec95031b6 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:57:06.287396 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fdebf0-48fd-4621-baf4-2d4ec95031b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:57:06.312848 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-26fdebf0-48fd-4621-baf4-2d4ec95031b6 None None] MFA Rules not processed for user `d4a0edb5f7e047bd9e669ab9ce91fac7`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:57:06.314599 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-26fdebf0-48fd-4621-baf4-2d4ec95031b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:57:06.383486 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1307/2620] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:57:06 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:57:06.396226 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68f089d-4a96-4a8a-ae5e-fb5d4159c86d admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:57:06.397952 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-e68f089d-4a96-4a8a-ae5e-fb5d4159c86d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:57:06.408853 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:57:06.455222 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68f089d-4a96-4a8a-ae5e-fb5d4159c86d admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:57:06.459324 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b49365-d5b9-4f52-b862-839f744e9e0c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecf4db289fba4e4a806198c62d31f7c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:57:06.460356 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b49365-d5b9-4f52-b862-839f744e9e0c service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:57:06.460680 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b49365-d5b9-4f52-b862-839f744e9e0c service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:57:06.460960 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b49365-d5b9-4f52-b862-839f744e9e0c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:57:06.469944 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b49365-d5b9-4f52-b862-839f744e9e0c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:57:06.471474 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b49365-d5b9-4f52-b862-839f744e9e0c service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:57:06.481040 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1314/2621] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:57:06 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:57:06.577528 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fdebf0-48fd-4621-baf4-2d4ec95031b6 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:57:06.579070 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-26fdebf0-48fd-4621-baf4-2d4ec95031b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:57:06.591836 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fdebf0-48fd-4621-baf4-2d4ec95031b6 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:57:06.595527 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67743a30-2133-4c9e-9e48-e1df3353cbdf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecf4db289fba4e4a806198c62d31f7c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:57:06.596150 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67743a30-2133-4c9e-9e48-e1df3353cbdf service cinder] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:57:06.596437 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67743a30-2133-4c9e-9e48-e1df3353cbdf service cinder] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:57:06.596779 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67743a30-2133-4c9e-9e48-e1df3353cbdf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:57:06.653920 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67743a30-2133-4c9e-9e48-e1df3353cbdf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:57:06.655414 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67743a30-2133-4c9e-9e48-e1df3353cbdf service cinder] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:57:06.664702 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1308/2622] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:57:06 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:57:10.870839 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b49365-d5b9-4f52-b862-839f744e9e0c service cinder] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:57:10.872325 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-76b49365-d5b9-4f52-b862-839f744e9e0c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:57:10.944124 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b49365-d5b9-4f52-b862-839f744e9e0c service cinder] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:57:10.949694 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9177ffb7-49eb-4552-9843-512f57fdd103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:57:10.949819 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9177ffb7-49eb-4552-9843-512f57fdd103 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:57:10.950081 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9177ffb7-49eb-4552-9843-512f57fdd103 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:57:10.950352 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9177ffb7-49eb-4552-9843-512f57fdd103 admin admin] PATH_INFO: `/v3/users/f8cfed4bb67e4bc78072ec3554879771` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:57:10.951591 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9177ffb7-49eb-4552-9843-512f57fdd103 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8cfed4bb67e4bc78072ec3554879771)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:57:10.953536 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9177ffb7-49eb-4552-9843-512f57fdd103 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:57:11.148294 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-9177ffb7-49eb-4552-9843-512f57fdd103 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8cfed4bb67e4bc78072ec3554879771'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:57:11.148882 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-9177ffb7-49eb-4552-9843-512f57fdd103 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8cfed4bb67e4bc78072ec3554879771'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:57:11.149487 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-9177ffb7-49eb-4552-9843-512f57fdd103 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8cfed4bb67e4bc78072ec3554879771'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:57:11.156250 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-9177ffb7-49eb-4552-9843-512f57fdd103 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8cfed4bb67e4bc78072ec3554879771'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:57:11.237572 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-9177ffb7-49eb-4552-9843-512f57fdd103 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8cfed4bb67e4bc78072ec3554879771'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:57:11.257075 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1315/2623] 10.222.0.81 () {58 vars in 1223 bytes} [Wed Jul 20 22:57:10 2022] DELETE /identity/v3/users/f8cfed4bb67e4bc78072ec3554879771 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:57:11.261962 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f1b36e-d05c-4f92-951c-e4e456b0b9f2 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:57:11.262434 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f1b36e-d05c-4f92-951c-e4e456b0b9f2 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:57:11.262553 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f1b36e-d05c-4f92-951c-e4e456b0b9f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:57:11.280725 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-b7f1b36e-d05c-4f92-951c-e4e456b0b9f2 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:57:11.281965 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-b7f1b36e-d05c-4f92-951c-e4e456b0b9f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:57:11.340763 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1309/2624] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:57:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:57:11.350550 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9177ffb7-49eb-4552-9843-512f57fdd103 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:57:11.356171 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-9177ffb7-49eb-4552-9843-512f57fdd103 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:57:11.390237 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:57:11.414638 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9177ffb7-49eb-4552-9843-512f57fdd103 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:57:11.418248 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ef5a97-cd73-430f-a656-5c24bf3a3109 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:57:11.419115 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ef5a97-cd73-430f-a656-5c24bf3a3109 service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:57:11.419408 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ef5a97-cd73-430f-a656-5c24bf3a3109 service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:57:11.419688 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ef5a97-cd73-430f-a656-5c24bf3a3109 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:57:11.429182 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ef5a97-cd73-430f-a656-5c24bf3a3109 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:57:11.431072 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ef5a97-cd73-430f-a656-5c24bf3a3109 service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:57:11.440897 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1316/2625] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:57:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:57:11.504681 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f1b36e-d05c-4f92-951c-e4e456b0b9f2 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:57:11.506251 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-b7f1b36e-d05c-4f92-951c-e4e456b0b9f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:57:11.515208 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:57:11.520692 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f1b36e-d05c-4f92-951c-e4e456b0b9f2 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:57:11.525266 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f51af0-d1be-4392-96c3-10fcd2bf81a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:57:11.526257 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f51af0-d1be-4392-96c3-10fcd2bf81a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:57:11.526608 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f51af0-d1be-4392-96c3-10fcd2bf81a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:57:11.526951 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f51af0-d1be-4392-96c3-10fcd2bf81a3 admin admin] PATH_INFO: `/v3/users/5236f2caddd34a90a7853864de2bbac7` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:57:11.528237 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f51af0-d1be-4392-96c3-10fcd2bf81a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5236f2caddd34a90a7853864de2bbac7)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:57:11.530154 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f51af0-d1be-4392-96c3-10fcd2bf81a3 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:57:11.765125 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-78f51af0-d1be-4392-96c3-10fcd2bf81a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5236f2caddd34a90a7853864de2bbac7'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:57:11.765894 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-78f51af0-d1be-4392-96c3-10fcd2bf81a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5236f2caddd34a90a7853864de2bbac7'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:57:11.766518 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-78f51af0-d1be-4392-96c3-10fcd2bf81a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5236f2caddd34a90a7853864de2bbac7'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:57:11.776910 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-78f51af0-d1be-4392-96c3-10fcd2bf81a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5236f2caddd34a90a7853864de2bbac7'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:57:11.870847 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-78f51af0-d1be-4392-96c3-10fcd2bf81a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5236f2caddd34a90a7853864de2bbac7'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:57:11.878092 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ef5a97-cd73-430f-a656-5c24bf3a3109 service neutron] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:57:11.880262 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-36ef5a97-cd73-430f-a656-5c24bf3a3109 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:57:11.894776 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1310/2626] 10.222.0.81 () {58 vars in 1223 bytes} [Wed Jul 20 22:57:11 2022] DELETE /identity/v3/users/5236f2caddd34a90a7853864de2bbac7 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 22:57:11.894960 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ef5a97-cd73-430f-a656-5c24bf3a3109 service neutron] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:57:11.897665 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd36bea-a64f-4e2c-b495-f7c910996602 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:57:11.898034 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd36bea-a64f-4e2c-b495-f7c910996602 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:57:11.899927 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd36bea-a64f-4e2c-b495-f7c910996602 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:57:11.922697 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-4dd36bea-a64f-4e2c-b495-f7c910996602 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:57:11.924199 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-4dd36bea-a64f-4e2c-b495-f7c910996602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:57:11.955704 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a708098-c7b8-4be4-ad41-1b652091c2fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:57:11.956603 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a708098-c7b8-4be4-ad41-1b652091c2fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:57:11.956897 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a708098-c7b8-4be4-ad41-1b652091c2fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:57:11.957174 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a708098-c7b8-4be4-ad41-1b652091c2fb admin admin] PATH_INFO: `/v3/projects/fe3edc44b7ee47139e03ec1dc3787da8` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:57:11.958377 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a708098-c7b8-4be4-ad41-1b652091c2fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe3edc44b7ee47139e03ec1dc3787da8)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:57:11.960451 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a708098-c7b8-4be4-ad41-1b652091c2fb admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:57:11.985614 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1311/2627] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:57:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 22:57:11.995534 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd36bea-a64f-4e2c-b495-f7c910996602 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:57:11.996915 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-4dd36bea-a64f-4e2c-b495-f7c910996602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:57:12.053548 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd36bea-a64f-4e2c-b495-f7c910996602 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:57:12.057290 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6737f02d-7b56-42fe-a017-ee9c64bfdc6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:57:12.057923 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6737f02d-7b56-42fe-a017-ee9c64bfdc6a service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:57:12.058215 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6737f02d-7b56-42fe-a017-ee9c64bfdc6a service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:57:12.058498 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6737f02d-7b56-42fe-a017-ee9c64bfdc6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:57:12.067906 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6737f02d-7b56-42fe-a017-ee9c64bfdc6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:57:12.069345 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6737f02d-7b56-42fe-a017-ee9c64bfdc6a service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:57:12.073272 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-5a708098-c7b8-4be4-ad41-1b652091c2fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe3edc44b7ee47139e03ec1dc3787da8', 'request_id': 'req-5a708098-c7b8-4be4-ad41-1b652091c2fb'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:57:12.079034 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1312/2628] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:57:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 22:57:12.080828 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-5a708098-c7b8-4be4-ad41-1b652091c2fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe3edc44b7ee47139e03ec1dc3787da8', 'request_id': 'req-5a708098-c7b8-4be4-ad41-1b652091c2fb'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:57:12.163174 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-5a708098-c7b8-4be4-ad41-1b652091c2fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe3edc44b7ee47139e03ec1dc3787da8', 'request_id': 'req-5a708098-c7b8-4be4-ad41-1b652091c2fb'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:57:12.174869 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-5a708098-c7b8-4be4-ad41-1b652091c2fb admin admin] The token cache is being invalidate because project fe3edc44b7ee47139e03ec1dc3787da8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:57:12.175110 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-5a708098-c7b8-4be4-ad41-1b652091c2fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:57:12.175753 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-5a708098-c7b8-4be4-ad41-1b652091c2fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:57:12.177294 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1317/2629] 10.222.0.81 () {58 vars in 1232 bytes} [Wed Jul 20 22:57:11 2022] DELETE /identity/v3/projects/fe3edc44b7ee47139e03ec1dc3787da8 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 22:57:12.564870 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6737f02d-7b56-42fe-a017-ee9c64bfdc6a service neutron] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:57:12.566474 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-6737f02d-7b56-42fe-a017-ee9c64bfdc6a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:57:12.582921 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6737f02d-7b56-42fe-a017-ee9c64bfdc6a service neutron] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:57:12.595832 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:57:12.640376 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f7741f-87cf-4e37-aa7c-de2d484a11f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:57:12.641256 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f7741f-87cf-4e37-aa7c-de2d484a11f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:57:12.641545 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f7741f-87cf-4e37-aa7c-de2d484a11f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:57:12.641853 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f7741f-87cf-4e37-aa7c-de2d484a11f7 admin admin] PATH_INFO: `/v3/projects/1cbc2d06b7fb4a9a80f0fa9e19aaf21c` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:57:12.643114 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f7741f-87cf-4e37-aa7c-de2d484a11f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1cbc2d06b7fb4a9a80f0fa9e19aaf21c)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:57:12.645022 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f7741f-87cf-4e37-aa7c-de2d484a11f7 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:57:12.758888 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-08f7741f-87cf-4e37-aa7c-de2d484a11f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1cbc2d06b7fb4a9a80f0fa9e19aaf21c', 'request_id': 'req-08f7741f-87cf-4e37-aa7c-de2d484a11f7'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:57:12.836708 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-08f7741f-87cf-4e37-aa7c-de2d484a11f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1cbc2d06b7fb4a9a80f0fa9e19aaf21c', 'request_id': 'req-08f7741f-87cf-4e37-aa7c-de2d484a11f7'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:57:12.842656 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-08f7741f-87cf-4e37-aa7c-de2d484a11f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1cbc2d06b7fb4a9a80f0fa9e19aaf21c', 'request_id': 'req-08f7741f-87cf-4e37-aa7c-de2d484a11f7'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:57:12.853786 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-08f7741f-87cf-4e37-aa7c-de2d484a11f7 admin admin] The token cache is being invalidate because project 1cbc2d06b7fb4a9a80f0fa9e19aaf21c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:57:12.856783 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-08f7741f-87cf-4e37-aa7c-de2d484a11f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:57:12.856783 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-08f7741f-87cf-4e37-aa7c-de2d484a11f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:57:12.856783 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1313/2630] 10.222.0.81 () {58 vars in 1232 bytes} [Wed Jul 20 22:57:12 2022] DELETE /identity/v3/projects/1cbc2d06b7fb4a9a80f0fa9e19aaf21c => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:57:12.860223 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a708098-c7b8-4be4-ad41-1b652091c2fb admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:57:12.862274 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-5a708098-c7b8-4be4-ad41-1b652091c2fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:57:12.874747 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a708098-c7b8-4be4-ad41-1b652091c2fb admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:57:12.930485 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f39c362-4936-4a44-ae9f-e1dcea6bd489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:57:12.931401 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f39c362-4936-4a44-ae9f-e1dcea6bd489 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:57:12.931713 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f39c362-4936-4a44-ae9f-e1dcea6bd489 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:57:12.931991 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f39c362-4936-4a44-ae9f-e1dcea6bd489 admin admin] PATH_INFO: `/v3/users/d4a0edb5f7e047bd9e669ab9ce91fac7` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:57:12.933179 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f39c362-4936-4a44-ae9f-e1dcea6bd489 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4a0edb5f7e047bd9e669ab9ce91fac7)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:57:12.935131 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f39c362-4936-4a44-ae9f-e1dcea6bd489 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:57:12.940246 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:57:13.153686 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-4f39c362-4936-4a44-ae9f-e1dcea6bd489 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4a0edb5f7e047bd9e669ab9ce91fac7'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:57:13.154388 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-4f39c362-4936-4a44-ae9f-e1dcea6bd489 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4a0edb5f7e047bd9e669ab9ce91fac7'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:57:13.155028 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-4f39c362-4936-4a44-ae9f-e1dcea6bd489 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4a0edb5f7e047bd9e669ab9ce91fac7'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:57:13.162472 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-4f39c362-4936-4a44-ae9f-e1dcea6bd489 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4a0edb5f7e047bd9e669ab9ce91fac7'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:57:13.220236 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-4f39c362-4936-4a44-ae9f-e1dcea6bd489 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4a0edb5f7e047bd9e669ab9ce91fac7'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:57:13.238309 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1318/2631] 10.222.0.81 () {58 vars in 1223 bytes} [Wed Jul 20 22:57:12 2022] DELETE /identity/v3/users/d4a0edb5f7e047bd9e669ab9ce91fac7 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:57:13.650824 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f7741f-87cf-4e37-aa7c-de2d484a11f7 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:57:13.652326 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-08f7741f-87cf-4e37-aa7c-de2d484a11f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:57:13.665975 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f7741f-87cf-4e37-aa7c-de2d484a11f7 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:57:13.724739 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0864a0f-601a-468f-8a96-831825347480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:57:13.725544 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0864a0f-601a-468f-8a96-831825347480 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:57:13.725830 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0864a0f-601a-468f-8a96-831825347480 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:57:13.726143 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0864a0f-601a-468f-8a96-831825347480 admin admin] PATH_INFO: `/v3/projects/b56b868f8933464db7abb77435e5d095` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:57:13.727388 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0864a0f-601a-468f-8a96-831825347480 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b56b868f8933464db7abb77435e5d095)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:57:13.729347 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0864a0f-601a-468f-8a96-831825347480 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:57:13.837588 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-e0864a0f-601a-468f-8a96-831825347480 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b56b868f8933464db7abb77435e5d095', 'request_id': 'req-e0864a0f-601a-468f-8a96-831825347480'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:57:13.839187 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:57:13.945279 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-e0864a0f-601a-468f-8a96-831825347480 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b56b868f8933464db7abb77435e5d095', 'request_id': 'req-e0864a0f-601a-468f-8a96-831825347480'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:57:13.951064 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-e0864a0f-601a-468f-8a96-831825347480 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b56b868f8933464db7abb77435e5d095', 'request_id': 'req-e0864a0f-601a-468f-8a96-831825347480'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:57:13.962643 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-e0864a0f-601a-468f-8a96-831825347480 admin admin] The token cache is being invalidate because project b56b868f8933464db7abb77435e5d095 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:57:13.962932 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-e0864a0f-601a-468f-8a96-831825347480 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:57:13.963588 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-e0864a0f-601a-468f-8a96-831825347480 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:57:13.965108 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1314/2632] 10.222.0.81 () {58 vars in 1232 bytes} [Wed Jul 20 22:57:13 2022] DELETE /identity/v3/projects/b56b868f8933464db7abb77435e5d095 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:57:13.970126 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d5086f-5b8b-48a1-9396-9251f1adb169 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:57:13.970583 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d5086f-5b8b-48a1-9396-9251f1adb169 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:57:13.970752 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d5086f-5b8b-48a1-9396-9251f1adb169 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:57:13.989424 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-c5d5086f-5b8b-48a1-9396-9251f1adb169 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:57:13.989424 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-c5d5086f-5b8b-48a1-9396-9251f1adb169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:57:14.047707 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1319/2633] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:57:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:57:14.061912 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c83699d-4560-4053-96e4-62611e491d3d None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:57:14.062203 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c83699d-4560-4053-96e4-62611e491d3d None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:57:14.062492 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c83699d-4560-4053-96e4-62611e491d3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:57:14.082832 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-5c83699d-4560-4053-96e4-62611e491d3d None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:57:14.084158 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-5c83699d-4560-4053-96e4-62611e491d3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:57:14.092514 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1315/2634] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:57:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 22:57:14.098013 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d5086f-5b8b-48a1-9396-9251f1adb169 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:57:14.109696 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d5086f-5b8b-48a1-9396-9251f1adb169 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:57:14.115516 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcb0171-a66f-461f-9aee-b562255d87f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:57:14.116451 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcb0171-a66f-461f-9aee-b562255d87f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:57:14.116837 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcb0171-a66f-461f-9aee-b562255d87f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:57:14.117240 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcb0171-a66f-461f-9aee-b562255d87f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:57:14.118351 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcb0171-a66f-461f-9aee-b562255d87f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:57:14.118710 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcb0171-a66f-461f-9aee-b562255d87f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:57:14.120341 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcb0171-a66f-461f-9aee-b562255d87f3 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:57:14.134139 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1320/2635] 10.222.0.81 () {58 vars in 1152 bytes} [Wed Jul 20 22:57:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:57:14.138727 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c83699d-4560-4053-96e4-62611e491d3d None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:57:14.147850 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c83699d-4560-4053-96e4-62611e491d3d None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:57:14.152576 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a775be-b2b5-45bd-b98d-8e5480516d5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:57:14.153218 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a775be-b2b5-45bd-b98d-8e5480516d5f admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:57:14.153500 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a775be-b2b5-45bd-b98d-8e5480516d5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:57:14.153773 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a775be-b2b5-45bd-b98d-8e5480516d5f admin admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:57:14.154462 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a775be-b2b5-45bd-b98d-8e5480516d5f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:57:14.156411 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a775be-b2b5-45bd-b98d-8e5480516d5f admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:57:14.239271 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1316/2636] 10.222.0.81 () {60 vars in 1152 bytes} [Wed Jul 20 22:57:14 2022] POST /identity/v3/projects => generated 385 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:57:14.243407 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcb0171-a66f-461f-9aee-b562255d87f3 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:57:14.253879 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcb0171-a66f-461f-9aee-b562255d87f3 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:57:14.309382 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58198069-2cff-4221-8f5d-0fa03177846f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:57:14.310052 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58198069-2cff-4221-8f5d-0fa03177846f admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:57:14.310345 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58198069-2cff-4221-8f5d-0fa03177846f admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:57:14.310627 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58198069-2cff-4221-8f5d-0fa03177846f admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:57:14.311414 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58198069-2cff-4221-8f5d-0fa03177846f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:57:14.313333 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58198069-2cff-4221-8f5d-0fa03177846f admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:57:14.416046 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:57:14.421817 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1321/2637] 10.222.0.81 () {60 vars in 1143 bytes} [Wed Jul 20 22:57:14 2022] POST /identity/v3/users => generated 293 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:57:14.426855 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a775be-b2b5-45bd-b98d-8e5480516d5f admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:57:14.437240 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a775be-b2b5-45bd-b98d-8e5480516d5f admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:57:14.442620 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16f4f79-5e3a-41bd-93cd-f516da745d41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:57:14.443310 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16f4f79-5e3a-41bd-93cd-f516da745d41 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:57:14.443595 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16f4f79-5e3a-41bd-93cd-f516da745d41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:57:14.443875 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16f4f79-5e3a-41bd-93cd-f516da745d41 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:57:14.444599 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16f4f79-5e3a-41bd-93cd-f516da745d41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:57:14.446228 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16f4f79-5e3a-41bd-93cd-f516da745d41 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:57:14.456612 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1317/2638] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:57:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:57:14.460792 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58198069-2cff-4221-8f5d-0fa03177846f admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:57:14.471254 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58198069-2cff-4221-8f5d-0fa03177846f admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:57:14.476639 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fa8922-5ddd-4241-a904-eb75d55d2302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:57:14.480714 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fa8922-5ddd-4241-a904-eb75d55d2302 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:57:14.480714 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fa8922-5ddd-4241-a904-eb75d55d2302 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:57:14.480714 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fa8922-5ddd-4241-a904-eb75d55d2302 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:57:14.480714 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fa8922-5ddd-4241-a904-eb75d55d2302 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:57:14.480714 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fa8922-5ddd-4241-a904-eb75d55d2302 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:57:14.491683 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1322/2639] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:57:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:57:14.495886 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16f4f79-5e3a-41bd-93cd-f516da745d41 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:57:14.505732 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16f4f79-5e3a-41bd-93cd-f516da745d41 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:57:14.510683 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b474e43b-346c-48d8-a9f9-6f5cf54b393a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:57:14.511519 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b474e43b-346c-48d8-a9f9-6f5cf54b393a admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:57:14.511801 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b474e43b-346c-48d8-a9f9-6f5cf54b393a admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:57:14.512075 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b474e43b-346c-48d8-a9f9-6f5cf54b393a admin admin] PATH_INFO: `/v3/projects/4ef10ce83b81403da1cfc65a510bc257/users/ab577e6b7b81434a8b0981c310b2cecb/roles/559942c4de1744169f8b5e0404ecde6b` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:57:14.526234 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b474e43b-346c-48d8-a9f9-6f5cf54b393a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ef10ce83b81403da1cfc65a510bc257, user_id=ab577e6b7b81434a8b0981c310b2cecb, role_id=559942c4de1744169f8b5e0404ecde6b)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:57:14.528913 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b474e43b-346c-48d8-a9f9-6f5cf54b393a admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:57:14.608400 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1318/2640] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:57:14 2022] PUT /identity/v3/projects/4ef10ce83b81403da1cfc65a510bc257/users/ab577e6b7b81434a8b0981c310b2cecb/roles/559942c4de1744169f8b5e0404ecde6b => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:57:14.619412 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87ce58e-1b4a-463e-9517-4f4fc7f2867e None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:57:14.619712 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87ce58e-1b4a-463e-9517-4f4fc7f2867e None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:57:14.619998 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87ce58e-1b4a-463e-9517-4f4fc7f2867e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:57:14.650052 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-b87ce58e-1b4a-463e-9517-4f4fc7f2867e None None] MFA Rules not processed for user `ab577e6b7b81434a8b0981c310b2cecb`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:57:14.651324 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-b87ce58e-1b4a-463e-9517-4f4fc7f2867e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:57:14.713898 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1323/2641] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:57:14 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:57:14.725032 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b474e43b-346c-48d8-a9f9-6f5cf54b393a admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:57:14.726753 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-b474e43b-346c-48d8-a9f9-6f5cf54b393a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:57:14.782193 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b474e43b-346c-48d8-a9f9-6f5cf54b393a admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:57:14.786226 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabd3445-c892-41d9-bdb5-023fea72aaae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecf4db289fba4e4a806198c62d31f7c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:57:14.786867 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabd3445-c892-41d9-bdb5-023fea72aaae service cinder] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:57:14.787189 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabd3445-c892-41d9-bdb5-023fea72aaae service cinder] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:57:14.787518 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabd3445-c892-41d9-bdb5-023fea72aaae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:57:14.795634 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabd3445-c892-41d9-bdb5-023fea72aaae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:57:14.797050 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabd3445-c892-41d9-bdb5-023fea72aaae service cinder] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:57:14.806345 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1319/2642] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:57:14 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:57:24.491312 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87ce58e-1b4a-463e-9517-4f4fc7f2867e None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:57:24.549577 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87ce58e-1b4a-463e-9517-4f4fc7f2867e None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:57:24.554147 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1d4f3f-1a59-4bb1-ad2d-ff8c43ca48c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:57:24.555105 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1d4f3f-1a59-4bb1-ad2d-ff8c43ca48c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:57:24.555420 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1d4f3f-1a59-4bb1-ad2d-ff8c43ca48c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:57:24.555697 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1d4f3f-1a59-4bb1-ad2d-ff8c43ca48c2 admin admin] PATH_INFO: `/v3/users/ab577e6b7b81434a8b0981c310b2cecb` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:57:24.556871 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1d4f3f-1a59-4bb1-ad2d-ff8c43ca48c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab577e6b7b81434a8b0981c310b2cecb)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:57:24.558761 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1d4f3f-1a59-4bb1-ad2d-ff8c43ca48c2 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:57:24.770689 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-1d1d4f3f-1a59-4bb1-ad2d-ff8c43ca48c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab577e6b7b81434a8b0981c310b2cecb'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:57:24.771329 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-1d1d4f3f-1a59-4bb1-ad2d-ff8c43ca48c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab577e6b7b81434a8b0981c310b2cecb'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:57:24.771948 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-1d1d4f3f-1a59-4bb1-ad2d-ff8c43ca48c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab577e6b7b81434a8b0981c310b2cecb'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:57:24.778940 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-1d1d4f3f-1a59-4bb1-ad2d-ff8c43ca48c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab577e6b7b81434a8b0981c310b2cecb'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:57:24.802369 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:INSERT=3,keystone:DELETE=3,keystone:SELECT=59 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:57:24.843041 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-1d1d4f3f-1a59-4bb1-ad2d-ff8c43ca48c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab577e6b7b81434a8b0981c310b2cecb'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:57:24.858541 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:57:24.864825 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1324/2643] 10.222.0.81 () {58 vars in 1223 bytes} [Wed Jul 20 22:57:24 2022] DELETE /identity/v3/users/ab577e6b7b81434a8b0981c310b2cecb => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:57:24.870326 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb5ee1b-aa96-4df7-8605-9cc5c4491565 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:57:24.870621 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb5ee1b-aa96-4df7-8605-9cc5c4491565 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:57:24.870904 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb5ee1b-aa96-4df7-8605-9cc5c4491565 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:57:24.888290 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-cfb5ee1b-aa96-4df7-8605-9cc5c4491565 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:57:24.889533 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-cfb5ee1b-aa96-4df7-8605-9cc5c4491565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:57:24.947632 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1320/2644] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:57:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:57:24.957956 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1d4f3f-1a59-4bb1-ad2d-ff8c43ca48c2 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:57:24.959398 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-1d1d4f3f-1a59-4bb1-ad2d-ff8c43ca48c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:57:25.016225 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1d4f3f-1a59-4bb1-ad2d-ff8c43ca48c2 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:57:25.020490 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8c67f8-4f10-467e-a31a-2aa6084528f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:57:25.021193 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8c67f8-4f10-467e-a31a-2aa6084528f6 service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:57:25.021423 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8c67f8-4f10-467e-a31a-2aa6084528f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:57:25.021706 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8c67f8-4f10-467e-a31a-2aa6084528f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:57:25.030827 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8c67f8-4f10-467e-a31a-2aa6084528f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:57:25.032322 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8c67f8-4f10-467e-a31a-2aa6084528f6 service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:57:25.041813 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1325/2645] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:57:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:57:25.496659 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb5ee1b-aa96-4df7-8605-9cc5c4491565 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:57:25.504848 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-cfb5ee1b-aa96-4df7-8605-9cc5c4491565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:57:25.515511 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb5ee1b-aa96-4df7-8605-9cc5c4491565 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:57:25.520437 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23d35b3-0b61-49cd-9a62-290754ac0fef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:57:25.521354 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23d35b3-0b61-49cd-9a62-290754ac0fef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:57:25.521646 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23d35b3-0b61-49cd-9a62-290754ac0fef admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:57:25.521926 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23d35b3-0b61-49cd-9a62-290754ac0fef admin admin] PATH_INFO: `/v3/projects/4ef10ce83b81403da1cfc65a510bc257` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:57:25.523153 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23d35b3-0b61-49cd-9a62-290754ac0fef admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ef10ce83b81403da1cfc65a510bc257)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:57:25.525068 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23d35b3-0b61-49cd-9a62-290754ac0fef admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:57:25.659894 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-d23d35b3-0b61-49cd-9a62-290754ac0fef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ef10ce83b81403da1cfc65a510bc257', 'request_id': 'req-d23d35b3-0b61-49cd-9a62-290754ac0fef'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:57:25.756740 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-d23d35b3-0b61-49cd-9a62-290754ac0fef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ef10ce83b81403da1cfc65a510bc257', 'request_id': 'req-d23d35b3-0b61-49cd-9a62-290754ac0fef'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:57:25.762918 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-d23d35b3-0b61-49cd-9a62-290754ac0fef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ef10ce83b81403da1cfc65a510bc257', 'request_id': 'req-d23d35b3-0b61-49cd-9a62-290754ac0fef'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:57:25.774696 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-d23d35b3-0b61-49cd-9a62-290754ac0fef admin admin] The token cache is being invalidate because project 4ef10ce83b81403da1cfc65a510bc257 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:57:25.774987 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-d23d35b3-0b61-49cd-9a62-290754ac0fef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:57:25.775663 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-d23d35b3-0b61-49cd-9a62-290754ac0fef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:57:25.777201 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1321/2646] 10.222.0.81 () {58 vars in 1232 bytes} [Wed Jul 20 22:57:25 2022] DELETE /identity/v3/projects/4ef10ce83b81403da1cfc65a510bc257 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:57:25.782396 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac5432d-500a-433d-b286-0f976a2f98e8 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:57:25.782887 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac5432d-500a-433d-b286-0f976a2f98e8 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:57:25.782985 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac5432d-500a-433d-b286-0f976a2f98e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:57:25.800172 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-9ac5432d-500a-433d-b286-0f976a2f98e8 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:57:25.801378 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-9ac5432d-500a-433d-b286-0f976a2f98e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:57:25.861830 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1326/2647] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:57:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:57:25.870551 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c575b5-e865-4e64-873d-96e3b0a139f7 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:57:25.871075 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c575b5-e865-4e64-873d-96e3b0a139f7 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:57:25.871191 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c575b5-e865-4e64-873d-96e3b0a139f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:57:25.890017 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-e9c575b5-e865-4e64-873d-96e3b0a139f7 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:57:25.891344 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-e9c575b5-e865-4e64-873d-96e3b0a139f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:57:25.900079 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1322/2648] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:57:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:57:25.904322 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac5432d-500a-433d-b286-0f976a2f98e8 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:57:25.914066 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac5432d-500a-433d-b286-0f976a2f98e8 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:57:25.918586 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf79fea-3bc4-4f0a-9cfc-d556a1dbc824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:57:25.919279 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf79fea-3bc4-4f0a-9cfc-d556a1dbc824 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:57:25.919568 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf79fea-3bc4-4f0a-9cfc-d556a1dbc824 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:57:25.919869 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf79fea-3bc4-4f0a-9cfc-d556a1dbc824 admin admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:57:25.920626 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf79fea-3bc4-4f0a-9cfc-d556a1dbc824 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:57:25.920936 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf79fea-3bc4-4f0a-9cfc-d556a1dbc824 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:57:25.922523 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf79fea-3bc4-4f0a-9cfc-d556a1dbc824 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:57:25.934334 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1327/2649] 10.222.0.81 () {58 vars in 1152 bytes} [Wed Jul 20 22:57:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:57:25.939737 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c575b5-e865-4e64-873d-96e3b0a139f7 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:57:25.949068 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c575b5-e865-4e64-873d-96e3b0a139f7 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:57:25.953778 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3418b284-a123-4f7a-ac1a-7e902daebf30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:57:25.954535 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3418b284-a123-4f7a-ac1a-7e902daebf30 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:57:25.954814 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3418b284-a123-4f7a-ac1a-7e902daebf30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:57:25.955662 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3418b284-a123-4f7a-ac1a-7e902daebf30 admin admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:57:25.956346 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3418b284-a123-4f7a-ac1a-7e902daebf30 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:57:25.958268 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3418b284-a123-4f7a-ac1a-7e902daebf30 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:57:26.026573 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1323/2650] 10.222.0.81 () {60 vars in 1152 bytes} [Wed Jul 20 22:57:25 2022] POST /identity/v3/projects => generated 373 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:57:26.030860 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf79fea-3bc4-4f0a-9cfc-d556a1dbc824 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:57:26.041027 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf79fea-3bc4-4f0a-9cfc-d556a1dbc824 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:57:26.096452 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57160540-f16a-416c-a11c-87289d1f3364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:57:26.097126 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57160540-f16a-416c-a11c-87289d1f3364 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:57:26.097413 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57160540-f16a-416c-a11c-87289d1f3364 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:57:26.097692 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57160540-f16a-416c-a11c-87289d1f3364 admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:57:26.098373 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57160540-f16a-416c-a11c-87289d1f3364 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:57:26.100296 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57160540-f16a-416c-a11c-87289d1f3364 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:57:26.156740 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1328/2651] 10.222.0.81 () {60 vars in 1143 bytes} [Wed Jul 20 22:57:26 2022] POST /identity/v3/users => generated 287 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:57:26.160964 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3418b284-a123-4f7a-ac1a-7e902daebf30 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:57:26.171228 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3418b284-a123-4f7a-ac1a-7e902daebf30 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:57:26.176264 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d64a06-5814-43e1-9897-0cb3b81efa36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:57:26.176925 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d64a06-5814-43e1-9897-0cb3b81efa36 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:57:26.177209 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d64a06-5814-43e1-9897-0cb3b81efa36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:57:26.177489 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d64a06-5814-43e1-9897-0cb3b81efa36 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:57:26.178217 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d64a06-5814-43e1-9897-0cb3b81efa36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:57:26.179856 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d64a06-5814-43e1-9897-0cb3b81efa36 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:57:26.194047 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1324/2652] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:57:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:57:26.195293 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57160540-f16a-416c-a11c-87289d1f3364 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:57:26.205325 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57160540-f16a-416c-a11c-87289d1f3364 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:57:26.209993 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3437beba-4c36-4fb6-a67d-1e2b1e7ea625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:57:26.210662 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3437beba-4c36-4fb6-a67d-1e2b1e7ea625 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:57:26.210955 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3437beba-4c36-4fb6-a67d-1e2b1e7ea625 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:57:26.211271 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3437beba-4c36-4fb6-a67d-1e2b1e7ea625 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:57:26.212005 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3437beba-4c36-4fb6-a67d-1e2b1e7ea625 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:57:26.213589 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3437beba-4c36-4fb6-a67d-1e2b1e7ea625 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:57:26.223446 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1329/2653] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:57:26 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:57:26.227343 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d64a06-5814-43e1-9897-0cb3b81efa36 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:57:26.237574 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d64a06-5814-43e1-9897-0cb3b81efa36 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:57:26.242194 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12a602a-8b00-4f5f-90c4-f15eebef86c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:57:26.243107 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12a602a-8b00-4f5f-90c4-f15eebef86c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:57:26.243388 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12a602a-8b00-4f5f-90c4-f15eebef86c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:57:26.243666 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12a602a-8b00-4f5f-90c4-f15eebef86c1 admin admin] PATH_INFO: `/v3/projects/52ccd4333faa4b21946b1cf7224e5020/users/3d47677b87a94491845f62f5847e107f/roles/559942c4de1744169f8b5e0404ecde6b` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:57:26.259599 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12a602a-8b00-4f5f-90c4-f15eebef86c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=52ccd4333faa4b21946b1cf7224e5020, user_id=3d47677b87a94491845f62f5847e107f, role_id=559942c4de1744169f8b5e0404ecde6b)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:57:26.262573 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12a602a-8b00-4f5f-90c4-f15eebef86c1 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:57:26.328091 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1325/2654] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:57:26 2022] PUT /identity/v3/projects/52ccd4333faa4b21946b1cf7224e5020/users/3d47677b87a94491845f62f5847e107f/roles/559942c4de1744169f8b5e0404ecde6b => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 22:57:26.338477 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23423689-778e-41be-9351-f7c22fd897f7 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:57:26.347454 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23423689-778e-41be-9351-f7c22fd897f7 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:57:26.347454 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23423689-778e-41be-9351-f7c22fd897f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:57:26.351656 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:57:26.365770 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-23423689-778e-41be-9351-f7c22fd897f7 None None] MFA Rules not processed for user `3d47677b87a94491845f62f5847e107f`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:57:26.366968 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-23423689-778e-41be-9351-f7c22fd897f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:57:26.432088 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1330/2655] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:57:26 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:57:26.436202 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12a602a-8b00-4f5f-90c4-f15eebef86c1 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:57:26.457934 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:57:26.498548 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12a602a-8b00-4f5f-90c4-f15eebef86c1 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:57:26.503845 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919d3a46-6f44-4da9-9207-ae826c643daf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:57:26.504505 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919d3a46-6f44-4da9-9207-ae826c643daf admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:57:26.504791 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919d3a46-6f44-4da9-9207-ae826c643daf admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:57:26.505071 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919d3a46-6f44-4da9-9207-ae826c643daf admin admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:57:26.505766 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919d3a46-6f44-4da9-9207-ae826c643daf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:57:26.507704 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919d3a46-6f44-4da9-9207-ae826c643daf admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:57:26.598535 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1326/2656] 10.222.0.81 () {60 vars in 1152 bytes} [Wed Jul 20 22:57:26 2022] POST /identity/v3/projects => generated 373 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:57:26.602714 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23423689-778e-41be-9351-f7c22fd897f7 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:57:26.613962 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23423689-778e-41be-9351-f7c22fd897f7 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:57:26.670534 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423a717a-99c1-4192-920b-970e4fbed46d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:57:26.671239 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423a717a-99c1-4192-920b-970e4fbed46d admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:57:26.671530 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423a717a-99c1-4192-920b-970e4fbed46d admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:57:26.671812 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423a717a-99c1-4192-920b-970e4fbed46d admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:57:26.672508 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423a717a-99c1-4192-920b-970e4fbed46d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:57:26.675064 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423a717a-99c1-4192-920b-970e4fbed46d admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:57:26.756843 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1331/2657] 10.222.0.81 () {60 vars in 1143 bytes} [Wed Jul 20 22:57:26 2022] POST /identity/v3/users => generated 293 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:57:26.761212 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919d3a46-6f44-4da9-9207-ae826c643daf admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:57:26.771584 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919d3a46-6f44-4da9-9207-ae826c643daf admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:57:26.776253 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332e4caa-d750-429e-bc54-66a5a9d6fbd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:57:26.776915 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332e4caa-d750-429e-bc54-66a5a9d6fbd5 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:57:26.777202 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332e4caa-d750-429e-bc54-66a5a9d6fbd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:57:26.777481 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332e4caa-d750-429e-bc54-66a5a9d6fbd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:57:26.778234 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332e4caa-d750-429e-bc54-66a5a9d6fbd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:57:26.779819 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332e4caa-d750-429e-bc54-66a5a9d6fbd5 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:57:26.790832 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1327/2658] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:57:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:57:26.794955 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423a717a-99c1-4192-920b-970e4fbed46d admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:57:26.806118 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423a717a-99c1-4192-920b-970e4fbed46d admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:57:26.810707 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca2831d-e742-48e1-a43c-755c22e56f6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:57:26.811635 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca2831d-e742-48e1-a43c-755c22e56f6a admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:57:26.811935 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca2831d-e742-48e1-a43c-755c22e56f6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:57:26.812203 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca2831d-e742-48e1-a43c-755c22e56f6a admin admin] PATH_INFO: `/v3/projects/ccb1456e21b643ccb606159388aafa64/users/94053e521df64355b91ec2f821ddcd40/roles/d89b6c92d35a401a8517f1d47f6be843` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:57:26.826402 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca2831d-e742-48e1-a43c-755c22e56f6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ccb1456e21b643ccb606159388aafa64, user_id=94053e521df64355b91ec2f821ddcd40, role_id=d89b6c92d35a401a8517f1d47f6be843)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:57:26.829571 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca2831d-e742-48e1-a43c-755c22e56f6a admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:57:26.905805 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1332/2659] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:57:26 2022] PUT /identity/v3/projects/ccb1456e21b643ccb606159388aafa64/users/94053e521df64355b91ec2f821ddcd40/roles/d89b6c92d35a401a8517f1d47f6be843 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:57:26.910581 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332e4caa-d750-429e-bc54-66a5a9d6fbd5 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:57:26.921597 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332e4caa-d750-429e-bc54-66a5a9d6fbd5 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:57:26.985942 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7dc8a6-7de3-4018-8e4f-ab00debb95d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:57:26.986617 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7dc8a6-7de3-4018-8e4f-ab00debb95d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:57:26.986907 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7dc8a6-7de3-4018-8e4f-ab00debb95d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:57:26.987218 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7dc8a6-7de3-4018-8e4f-ab00debb95d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:57:26.987943 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7dc8a6-7de3-4018-8e4f-ab00debb95d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:57:26.989551 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7dc8a6-7de3-4018-8e4f-ab00debb95d6 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:57:26.999158 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1328/2660] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:57:26 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:57:27.003304 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca2831d-e742-48e1-a43c-755c22e56f6a admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:57:27.013960 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca2831d-e742-48e1-a43c-755c22e56f6a admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:57:27.019257 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68534898-fc6d-4c76-8845-5425109806c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:57:27.020185 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68534898-fc6d-4c76-8845-5425109806c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:57:27.020470 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68534898-fc6d-4c76-8845-5425109806c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:57:27.020748 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68534898-fc6d-4c76-8845-5425109806c8 admin admin] PATH_INFO: `/v3/projects/ccb1456e21b643ccb606159388aafa64/users/94053e521df64355b91ec2f821ddcd40/roles/d89b6c92d35a401a8517f1d47f6be843` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:57:27.022926 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68534898-fc6d-4c76-8845-5425109806c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ccb1456e21b643ccb606159388aafa64, user_id=94053e521df64355b91ec2f821ddcd40, role_id=d89b6c92d35a401a8517f1d47f6be843)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:57:27.025566 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68534898-fc6d-4c76-8845-5425109806c8 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:57:27.037017 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1333/2661] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:57:27 2022] PUT /identity/v3/projects/ccb1456e21b643ccb606159388aafa64/users/94053e521df64355b91ec2f821ddcd40/roles/d89b6c92d35a401a8517f1d47f6be843 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:57:27.047383 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d3b405-bdc4-4152-8c5c-b6c9d9c3c3fe None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:57:27.047681 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d3b405-bdc4-4152-8c5c-b6c9d9c3c3fe None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:57:27.047971 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d3b405-bdc4-4152-8c5c-b6c9d9c3c3fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:57:27.073718 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-17d3b405-bdc4-4152-8c5c-b6c9d9c3c3fe None None] MFA Rules not processed for user `94053e521df64355b91ec2f821ddcd40`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:57:27.074938 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-17d3b405-bdc4-4152-8c5c-b6c9d9c3c3fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:57:27.143521 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1329/2662] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:57:27 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 22:57:27.154156 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68534898-fc6d-4c76-8845-5425109806c8 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:57:27.155880 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-68534898-fc6d-4c76-8845-5425109806c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:57:27.213674 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:57:27.217377 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68534898-fc6d-4c76-8845-5425109806c8 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:57:27.221123 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97621c6b-79ed-4d8f-b151-c9f3782c23fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecf4db289fba4e4a806198c62d31f7c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:57:27.221774 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97621c6b-79ed-4d8f-b151-c9f3782c23fc service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:57:27.222061 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97621c6b-79ed-4d8f-b151-c9f3782c23fc service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:57:27.222475 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97621c6b-79ed-4d8f-b151-c9f3782c23fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:57:27.283562 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97621c6b-79ed-4d8f-b151-c9f3782c23fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:57:27.284939 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97621c6b-79ed-4d8f-b151-c9f3782c23fc service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:57:27.293885 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1334/2663] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:57:27 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:57:31.441671 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d3b405-bdc4-4152-8c5c-b6c9d9c3c3fe None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:57:31.458988 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:57:31.502719 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d3b405-bdc4-4152-8c5c-b6c9d9c3c3fe None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:57:31.507295 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfff514-4428-4cd8-9a7c-59c9a18848ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:57:31.508194 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfff514-4428-4cd8-9a7c-59c9a18848ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:57:31.508486 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfff514-4428-4cd8-9a7c-59c9a18848ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:57:31.508769 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfff514-4428-4cd8-9a7c-59c9a18848ec admin admin] PATH_INFO: `/v3/users/3d47677b87a94491845f62f5847e107f` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:57:31.509998 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfff514-4428-4cd8-9a7c-59c9a18848ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d47677b87a94491845f62f5847e107f)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:57:31.512225 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfff514-4428-4cd8-9a7c-59c9a18848ec admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:57:31.691299 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-dbfff514-4428-4cd8-9a7c-59c9a18848ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d47677b87a94491845f62f5847e107f'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:57:31.691866 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-dbfff514-4428-4cd8-9a7c-59c9a18848ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d47677b87a94491845f62f5847e107f'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:57:31.692544 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-dbfff514-4428-4cd8-9a7c-59c9a18848ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d47677b87a94491845f62f5847e107f'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:57:31.699427 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-dbfff514-4428-4cd8-9a7c-59c9a18848ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d47677b87a94491845f62f5847e107f'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:57:31.779992 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-dbfff514-4428-4cd8-9a7c-59c9a18848ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d47677b87a94491845f62f5847e107f'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:57:31.798615 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1330/2664] 10.222.0.81 () {58 vars in 1223 bytes} [Wed Jul 20 22:57:31 2022] DELETE /identity/v3/users/3d47677b87a94491845f62f5847e107f => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:57:31.803525 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e990b2-46f5-4c31-a27e-10f583d057f4 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:57:31.803987 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e990b2-46f5-4c31-a27e-10f583d057f4 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:57:31.804114 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e990b2-46f5-4c31-a27e-10f583d057f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:57:31.821778 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-70e990b2-46f5-4c31-a27e-10f583d057f4 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:57:31.822971 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-70e990b2-46f5-4c31-a27e-10f583d057f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:57:31.886246 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1335/2665] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:57:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:57:31.896158 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfff514-4428-4cd8-9a7c-59c9a18848ec admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:57:31.897659 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-dbfff514-4428-4cd8-9a7c-59c9a18848ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:57:31.954672 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfff514-4428-4cd8-9a7c-59c9a18848ec admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:57:31.958252 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bda488-84e5-4254-8065-cf7b8d457f73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:57:31.958900 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bda488-84e5-4254-8065-cf7b8d457f73 service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:57:31.959222 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bda488-84e5-4254-8065-cf7b8d457f73 service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:57:31.959499 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bda488-84e5-4254-8065-cf7b8d457f73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:57:31.968455 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bda488-84e5-4254-8065-cf7b8d457f73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:57:31.970430 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bda488-84e5-4254-8065-cf7b8d457f73 service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:57:31.980053 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1331/2666] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:57:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:57:32.387338 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e990b2-46f5-4c31-a27e-10f583d057f4 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:57:32.388796 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-70e990b2-46f5-4c31-a27e-10f583d057f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:57:32.403622 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e990b2-46f5-4c31-a27e-10f583d057f4 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:57:32.409396 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa5c96a-1ea5-4160-8d95-77cb425587e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:57:32.410533 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa5c96a-1ea5-4160-8d95-77cb425587e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:57:32.410833 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa5c96a-1ea5-4160-8d95-77cb425587e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:57:32.411199 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa5c96a-1ea5-4160-8d95-77cb425587e8 admin admin] PATH_INFO: `/v3/projects/52ccd4333faa4b21946b1cf7224e5020` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:57:32.412336 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa5c96a-1ea5-4160-8d95-77cb425587e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=52ccd4333faa4b21946b1cf7224e5020)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:57:32.414475 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa5c96a-1ea5-4160-8d95-77cb425587e8 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:57:32.546070 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-baa5c96a-1ea5-4160-8d95-77cb425587e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52ccd4333faa4b21946b1cf7224e5020', 'request_id': 'req-baa5c96a-1ea5-4160-8d95-77cb425587e8'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:57:32.552323 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-baa5c96a-1ea5-4160-8d95-77cb425587e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52ccd4333faa4b21946b1cf7224e5020', 'request_id': 'req-baa5c96a-1ea5-4160-8d95-77cb425587e8'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:57:32.625476 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-baa5c96a-1ea5-4160-8d95-77cb425587e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52ccd4333faa4b21946b1cf7224e5020', 'request_id': 'req-baa5c96a-1ea5-4160-8d95-77cb425587e8'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:57:32.636747 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-baa5c96a-1ea5-4160-8d95-77cb425587e8 admin admin] The token cache is being invalidate because project 52ccd4333faa4b21946b1cf7224e5020 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:57:32.637039 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-baa5c96a-1ea5-4160-8d95-77cb425587e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:57:32.637690 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-baa5c96a-1ea5-4160-8d95-77cb425587e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:57:32.639317 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1336/2667] 10.222.0.81 () {58 vars in 1232 bytes} [Wed Jul 20 22:57:32 2022] DELETE /identity/v3/projects/52ccd4333faa4b21946b1cf7224e5020 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:57:32.643946 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bda488-84e5-4254-8065-cf7b8d457f73 service neutron] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:57:32.653671 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bda488-84e5-4254-8065-cf7b8d457f73 service neutron] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:57:32.686234 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:57:32.709114 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cf4969-ed53-46f5-9a51-bc0cb72e70c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:57:32.710034 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cf4969-ed53-46f5-9a51-bc0cb72e70c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:57:32.710323 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cf4969-ed53-46f5-9a51-bc0cb72e70c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:57:32.710631 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cf4969-ed53-46f5-9a51-bc0cb72e70c8 admin admin] PATH_INFO: `/v3/users/94053e521df64355b91ec2f821ddcd40` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:57:32.711890 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cf4969-ed53-46f5-9a51-bc0cb72e70c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=94053e521df64355b91ec2f821ddcd40)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:57:32.713813 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cf4969-ed53-46f5-9a51-bc0cb72e70c8 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:57:32.919305 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-d1cf4969-ed53-46f5-9a51-bc0cb72e70c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94053e521df64355b91ec2f821ddcd40'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:57:32.919990 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-d1cf4969-ed53-46f5-9a51-bc0cb72e70c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94053e521df64355b91ec2f821ddcd40'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:57:32.920652 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-d1cf4969-ed53-46f5-9a51-bc0cb72e70c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94053e521df64355b91ec2f821ddcd40'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:57:32.928274 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-d1cf4969-ed53-46f5-9a51-bc0cb72e70c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94053e521df64355b91ec2f821ddcd40'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:57:33.025568 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-d1cf4969-ed53-46f5-9a51-bc0cb72e70c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94053e521df64355b91ec2f821ddcd40'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:57:33.043881 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1332/2668] 10.222.0.81 () {58 vars in 1223 bytes} [Wed Jul 20 22:57:32 2022] DELETE /identity/v3/users/94053e521df64355b91ec2f821ddcd40 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 22:57:33.730782 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa5c96a-1ea5-4160-8d95-77cb425587e8 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:57:33.732340 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-baa5c96a-1ea5-4160-8d95-77cb425587e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:57:33.745802 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa5c96a-1ea5-4160-8d95-77cb425587e8 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:57:33.771298 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:57:33.802628 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f9d88a-a520-4711-9125-2164f9540e40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:57:33.803561 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f9d88a-a520-4711-9125-2164f9540e40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:57:33.803844 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f9d88a-a520-4711-9125-2164f9540e40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:57:33.804119 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f9d88a-a520-4711-9125-2164f9540e40 admin admin] PATH_INFO: `/v3/projects/ccb1456e21b643ccb606159388aafa64` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:57:33.805347 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f9d88a-a520-4711-9125-2164f9540e40 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ccb1456e21b643ccb606159388aafa64)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:57:33.807275 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f9d88a-a520-4711-9125-2164f9540e40 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:57:33.952138 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-36f9d88a-a520-4711-9125-2164f9540e40 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ccb1456e21b643ccb606159388aafa64', 'request_id': 'req-36f9d88a-a520-4711-9125-2164f9540e40'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:57:33.958100 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-36f9d88a-a520-4711-9125-2164f9540e40 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ccb1456e21b643ccb606159388aafa64', 'request_id': 'req-36f9d88a-a520-4711-9125-2164f9540e40'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:57:34.031993 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-36f9d88a-a520-4711-9125-2164f9540e40 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ccb1456e21b643ccb606159388aafa64', 'request_id': 'req-36f9d88a-a520-4711-9125-2164f9540e40'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:57:34.043628 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-36f9d88a-a520-4711-9125-2164f9540e40 admin admin] The token cache is being invalidate because project ccb1456e21b643ccb606159388aafa64 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:57:34.043923 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-36f9d88a-a520-4711-9125-2164f9540e40 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:57:34.044588 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-36f9d88a-a520-4711-9125-2164f9540e40 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:57:34.046103 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1337/2669] 10.222.0.81 () {58 vars in 1232 bytes} [Wed Jul 20 22:57:33 2022] DELETE /identity/v3/projects/ccb1456e21b643ccb606159388aafa64 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:57:34.051627 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4971aa6-f21d-4923-b1bc-2d204b667e25 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:57:34.052123 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4971aa6-f21d-4923-b1bc-2d204b667e25 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:57:34.052231 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4971aa6-f21d-4923-b1bc-2d204b667e25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:57:34.070217 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-c4971aa6-f21d-4923-b1bc-2d204b667e25 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:57:34.077580 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-c4971aa6-f21d-4923-b1bc-2d204b667e25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:57:34.131636 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1333/2670] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:57:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:57:34.140832 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd16385c-3186-4d37-969b-e9c9be1ec1d2 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:57:34.141131 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd16385c-3186-4d37-969b-e9c9be1ec1d2 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:57:34.141415 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd16385c-3186-4d37-969b-e9c9be1ec1d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:57:34.159755 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-dd16385c-3186-4d37-969b-e9c9be1ec1d2 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:57:34.160960 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-dd16385c-3186-4d37-969b-e9c9be1ec1d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:57:34.169381 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1338/2671] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:57:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:57:34.173825 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4971aa6-f21d-4923-b1bc-2d204b667e25 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:57:34.183449 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4971aa6-f21d-4923-b1bc-2d204b667e25 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:57:34.188015 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5467166-d3fc-4a45-98f0-a4a0d486dc13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:57:34.188672 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5467166-d3fc-4a45-98f0-a4a0d486dc13 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:57:34.188953 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5467166-d3fc-4a45-98f0-a4a0d486dc13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:57:34.189231 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5467166-d3fc-4a45-98f0-a4a0d486dc13 admin admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:57:34.189952 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5467166-d3fc-4a45-98f0-a4a0d486dc13 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:57:34.190284 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5467166-d3fc-4a45-98f0-a4a0d486dc13 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:57:34.191889 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5467166-d3fc-4a45-98f0-a4a0d486dc13 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:57:34.203666 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1334/2672] 10.222.0.81 () {58 vars in 1152 bytes} [Wed Jul 20 22:57:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:57:34.208068 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd16385c-3186-4d37-969b-e9c9be1ec1d2 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:57:34.217354 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd16385c-3186-4d37-969b-e9c9be1ec1d2 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:57:34.222557 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4ff7fa-9c31-42ff-aaa3-535e8d09f75c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:57:34.223271 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4ff7fa-9c31-42ff-aaa3-535e8d09f75c admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:57:34.223561 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4ff7fa-9c31-42ff-aaa3-535e8d09f75c admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:57:34.223839 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4ff7fa-9c31-42ff-aaa3-535e8d09f75c admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:57:34.224527 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4ff7fa-9c31-42ff-aaa3-535e8d09f75c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:57:34.226413 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4ff7fa-9c31-42ff-aaa3-535e8d09f75c admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:57:34.341235 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1339/2673] 10.222.0.81 () {60 vars in 1152 bytes} [Wed Jul 20 22:57:34 2022] POST /identity/v3/projects => generated 375 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 22:57:34.345280 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5467166-d3fc-4a45-98f0-a4a0d486dc13 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:57:34.355897 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5467166-d3fc-4a45-98f0-a4a0d486dc13 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:57:34.396807 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:57:34.412179 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f65142a-e81b-454a-9ec8-c9d47257101f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:57:34.412926 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f65142a-e81b-454a-9ec8-c9d47257101f admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:57:34.413215 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f65142a-e81b-454a-9ec8-c9d47257101f admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:57:34.413496 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f65142a-e81b-454a-9ec8-c9d47257101f admin admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:57:34.414197 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f65142a-e81b-454a-9ec8-c9d47257101f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:57:34.416184 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f65142a-e81b-454a-9ec8-c9d47257101f admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:57:34.545840 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1335/2674] 10.222.0.81 () {60 vars in 1143 bytes} [Wed Jul 20 22:57:34 2022] POST /identity/v3/users => generated 288 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:57:34.549926 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4ff7fa-9c31-42ff-aaa3-535e8d09f75c admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:57:34.560140 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4ff7fa-9c31-42ff-aaa3-535e8d09f75c admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:57:34.564794 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1f78d8-3b95-438a-ba6b-f3f58d311547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:57:34.565464 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1f78d8-3b95-438a-ba6b-f3f58d311547 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:57:34.565754 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1f78d8-3b95-438a-ba6b-f3f58d311547 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:57:34.566039 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1f78d8-3b95-438a-ba6b-f3f58d311547 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:57:34.566770 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1f78d8-3b95-438a-ba6b-f3f58d311547 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:57:34.568446 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1f78d8-3b95-438a-ba6b-f3f58d311547 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:57:34.578853 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1340/2675] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:57:34 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:57:34.583272 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f65142a-e81b-454a-9ec8-c9d47257101f admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:57:34.593043 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f65142a-e81b-454a-9ec8-c9d47257101f admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:57:34.598322 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d416e831-2cf6-4056-ac7a-38ecedeb4b6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:57:34.598978 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d416e831-2cf6-4056-ac7a-38ecedeb4b6d admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:57:34.599299 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d416e831-2cf6-4056-ac7a-38ecedeb4b6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:57:34.599582 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d416e831-2cf6-4056-ac7a-38ecedeb4b6d admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:57:34.600309 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d416e831-2cf6-4056-ac7a-38ecedeb4b6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:57:34.601941 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d416e831-2cf6-4056-ac7a-38ecedeb4b6d admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:57:34.612382 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1336/2676] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:57:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:57:34.616559 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1f78d8-3b95-438a-ba6b-f3f58d311547 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:57:34.626288 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1f78d8-3b95-438a-ba6b-f3f58d311547 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:57:34.630724 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df762661-ce3c-42c0-b149-42bc7720c2f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:57:34.631766 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df762661-ce3c-42c0-b149-42bc7720c2f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:57:34.632044 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df762661-ce3c-42c0-b149-42bc7720c2f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:57:34.632313 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df762661-ce3c-42c0-b149-42bc7720c2f5 admin admin] PATH_INFO: `/v3/projects/03fe823060304d7590927c2f6524fac3/users/664d2850bd3b494593fc44851ac01504/roles/559942c4de1744169f8b5e0404ecde6b` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:57:34.647669 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df762661-ce3c-42c0-b149-42bc7720c2f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=03fe823060304d7590927c2f6524fac3, user_id=664d2850bd3b494593fc44851ac01504, role_id=559942c4de1744169f8b5e0404ecde6b)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:57:34.650429 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df762661-ce3c-42c0-b149-42bc7720c2f5 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:57:34.808488 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1341/2677] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:57:34 2022] PUT /identity/v3/projects/03fe823060304d7590927c2f6524fac3/users/664d2850bd3b494593fc44851ac01504/roles/559942c4de1744169f8b5e0404ecde6b => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:57:34.819544 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7910f2a8-678f-41b4-af6b-efd1a61d4c60 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:57:34.819838 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7910f2a8-678f-41b4-af6b-efd1a61d4c60 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:57:34.820118 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7910f2a8-678f-41b4-af6b-efd1a61d4c60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:57:34.845858 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-7910f2a8-678f-41b4-af6b-efd1a61d4c60 None None] MFA Rules not processed for user `664d2850bd3b494593fc44851ac01504`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:57:34.847122 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-7910f2a8-678f-41b4-af6b-efd1a61d4c60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:57:34.909749 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1337/2678] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:57:34 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:57:34.914193 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df762661-ce3c-42c0-b149-42bc7720c2f5 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:57:34.965529 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:57:34.975705 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df762661-ce3c-42c0-b149-42bc7720c2f5 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:57:34.980422 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44afd8f7-2985-4b9b-97e0-c836bf86a3f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:57:34.981079 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44afd8f7-2985-4b9b-97e0-c836bf86a3f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:57:34.981364 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44afd8f7-2985-4b9b-97e0-c836bf86a3f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:57:34.981640 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44afd8f7-2985-4b9b-97e0-c836bf86a3f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:57:34.982321 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44afd8f7-2985-4b9b-97e0-c836bf86a3f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:57:34.984308 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44afd8f7-2985-4b9b-97e0-c836bf86a3f3 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:57:35.055611 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1342/2679] 10.222.0.81 () {60 vars in 1152 bytes} [Wed Jul 20 22:57:34 2022] POST /identity/v3/projects => generated 377 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:57:35.065064 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7910f2a8-678f-41b4-af6b-efd1a61d4c60 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:57:35.071135 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7910f2a8-678f-41b4-af6b-efd1a61d4c60 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:57:35.127133 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578f3a5e-6f17-4b80-b930-16ba7b894026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:57:35.127805 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578f3a5e-6f17-4b80-b930-16ba7b894026 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:57:35.128088 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578f3a5e-6f17-4b80-b930-16ba7b894026 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:57:35.128365 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578f3a5e-6f17-4b80-b930-16ba7b894026 admin admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:57:35.129041 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578f3a5e-6f17-4b80-b930-16ba7b894026 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:57:35.130938 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578f3a5e-6f17-4b80-b930-16ba7b894026 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:57:35.237611 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1338/2680] 10.222.0.81 () {60 vars in 1143 bytes} [Wed Jul 20 22:57:35 2022] POST /identity/v3/users => generated 289 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:57:35.241551 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44afd8f7-2985-4b9b-97e0-c836bf86a3f3 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:57:35.251668 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44afd8f7-2985-4b9b-97e0-c836bf86a3f3 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:57:35.256382 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be351b7-8946-47f3-9334-cf6d3ff587ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:57:35.257048 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be351b7-8946-47f3-9334-cf6d3ff587ba admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:57:35.257337 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be351b7-8946-47f3-9334-cf6d3ff587ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:57:35.257618 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be351b7-8946-47f3-9334-cf6d3ff587ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:57:35.258341 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be351b7-8946-47f3-9334-cf6d3ff587ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:57:35.259967 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be351b7-8946-47f3-9334-cf6d3ff587ba admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:57:35.271336 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1343/2681] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:57:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:57:35.274939 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578f3a5e-6f17-4b80-b930-16ba7b894026 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:57:35.285701 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578f3a5e-6f17-4b80-b930-16ba7b894026 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:57:35.290341 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f410fb2-b98f-4145-a9f9-3b0b1bb7274d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:57:35.290995 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f410fb2-b98f-4145-a9f9-3b0b1bb7274d admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:57:35.291311 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f410fb2-b98f-4145-a9f9-3b0b1bb7274d admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:57:35.291588 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f410fb2-b98f-4145-a9f9-3b0b1bb7274d admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:57:35.292308 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f410fb2-b98f-4145-a9f9-3b0b1bb7274d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:57:35.294541 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f410fb2-b98f-4145-a9f9-3b0b1bb7274d admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:57:35.304089 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1339/2682] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:57:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:57:35.308142 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be351b7-8946-47f3-9334-cf6d3ff587ba admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:57:35.318314 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be351b7-8946-47f3-9334-cf6d3ff587ba admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:57:35.322932 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f16047-9284-4d87-b5b8-f381ac54b9bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:57:35.323867 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f16047-9284-4d87-b5b8-f381ac54b9bc admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:57:35.324160 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f16047-9284-4d87-b5b8-f381ac54b9bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:57:35.324445 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f16047-9284-4d87-b5b8-f381ac54b9bc admin admin] PATH_INFO: `/v3/projects/d71193306ded43beb8136d938b52d2cb/users/269f6d05b2f2461eb97b398a9b3a40d9/roles/559942c4de1744169f8b5e0404ecde6b` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:57:35.339406 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f16047-9284-4d87-b5b8-f381ac54b9bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=d71193306ded43beb8136d938b52d2cb, user_id=269f6d05b2f2461eb97b398a9b3a40d9, role_id=559942c4de1744169f8b5e0404ecde6b)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:57:35.342031 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f16047-9284-4d87-b5b8-f381ac54b9bc admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:57:35.448627 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1344/2683] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:57:35 2022] PUT /identity/v3/projects/d71193306ded43beb8136d938b52d2cb/users/269f6d05b2f2461eb97b398a9b3a40d9/roles/559942c4de1744169f8b5e0404ecde6b => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:57:35.459248 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6205ebb-e8b1-4e49-a0c5-d2895f0f0cbb None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:57:35.459582 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6205ebb-e8b1-4e49-a0c5-d2895f0f0cbb None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:57:35.459873 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6205ebb-e8b1-4e49-a0c5-d2895f0f0cbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:57:35.488017 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-e6205ebb-e8b1-4e49-a0c5-d2895f0f0cbb None None] MFA Rules not processed for user `269f6d05b2f2461eb97b398a9b3a40d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:57:35.489224 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-e6205ebb-e8b1-4e49-a0c5-d2895f0f0cbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:57:35.506852 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:57:35.552455 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1340/2684] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:57:35 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:57:35.557085 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f16047-9284-4d87-b5b8-f381ac54b9bc admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:57:35.617724 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f16047-9284-4d87-b5b8-f381ac54b9bc admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:57:35.622209 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d784e14-e7bf-40f3-a269-d6986dcbf381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:57:35.622881 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d784e14-e7bf-40f3-a269-d6986dcbf381 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:57:35.623200 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d784e14-e7bf-40f3-a269-d6986dcbf381 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:57:35.623478 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d784e14-e7bf-40f3-a269-d6986dcbf381 admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:57:35.624154 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d784e14-e7bf-40f3-a269-d6986dcbf381 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:57:35.626054 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d784e14-e7bf-40f3-a269-d6986dcbf381 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:57:35.730803 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1345/2685] 10.222.0.81 () {60 vars in 1152 bytes} [Wed Jul 20 22:57:35 2022] POST /identity/v3/projects => generated 373 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:57:35.735112 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6205ebb-e8b1-4e49-a0c5-d2895f0f0cbb None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:57:35.747869 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6205ebb-e8b1-4e49-a0c5-d2895f0f0cbb None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:57:35.805288 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f136bb4-a5c2-43d6-a971-146c10d0d634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:57:35.805972 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f136bb4-a5c2-43d6-a971-146c10d0d634 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:57:35.806257 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f136bb4-a5c2-43d6-a971-146c10d0d634 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:57:35.806538 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f136bb4-a5c2-43d6-a971-146c10d0d634 admin admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:57:35.807263 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f136bb4-a5c2-43d6-a971-146c10d0d634 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:57:35.809192 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f136bb4-a5c2-43d6-a971-146c10d0d634 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:57:35.900485 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1341/2686] 10.222.0.81 () {60 vars in 1143 bytes} [Wed Jul 20 22:57:35 2022] POST /identity/v3/users => generated 293 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:57:35.904464 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d784e14-e7bf-40f3-a269-d6986dcbf381 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:57:35.916152 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d784e14-e7bf-40f3-a269-d6986dcbf381 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:57:35.921261 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08630c84-0357-4283-a4a6-9ac491719a6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:57:35.921939 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08630c84-0357-4283-a4a6-9ac491719a6b admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:57:35.922229 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08630c84-0357-4283-a4a6-9ac491719a6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:57:35.922552 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08630c84-0357-4283-a4a6-9ac491719a6b admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:57:35.923304 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08630c84-0357-4283-a4a6-9ac491719a6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:57:35.924908 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08630c84-0357-4283-a4a6-9ac491719a6b admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:57:35.936679 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1346/2687] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:57:35 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:57:35.940766 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f136bb4-a5c2-43d6-a971-146c10d0d634 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:57:35.950634 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f136bb4-a5c2-43d6-a971-146c10d0d634 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:57:35.955772 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3cac61-dcf3-47c2-a2f3-f3f426b884ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:57:35.956659 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3cac61-dcf3-47c2-a2f3-f3f426b884ea admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:57:35.956942 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3cac61-dcf3-47c2-a2f3-f3f426b884ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:57:35.957215 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3cac61-dcf3-47c2-a2f3-f3f426b884ea admin admin] PATH_INFO: `/v3/projects/136bb2bdfb31451fb45db10434e04b7f/users/ab87270f385c493faad0c61ea54cffa0/roles/d89b6c92d35a401a8517f1d47f6be843` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:57:35.972789 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3cac61-dcf3-47c2-a2f3-f3f426b884ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=136bb2bdfb31451fb45db10434e04b7f, user_id=ab87270f385c493faad0c61ea54cffa0, role_id=d89b6c92d35a401a8517f1d47f6be843)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:57:35.975462 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3cac61-dcf3-47c2-a2f3-f3f426b884ea admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:57:36.043661 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1342/2688] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:57:35 2022] PUT /identity/v3/projects/136bb2bdfb31451fb45db10434e04b7f/users/ab87270f385c493faad0c61ea54cffa0/roles/d89b6c92d35a401a8517f1d47f6be843 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:57:36.047922 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08630c84-0357-4283-a4a6-9ac491719a6b admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:57:36.058141 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08630c84-0357-4283-a4a6-9ac491719a6b admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:57:36.121161 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5eaf66-15da-4aad-9068-d75bac5cedac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:57:36.121864 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5eaf66-15da-4aad-9068-d75bac5cedac admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:57:36.122150 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5eaf66-15da-4aad-9068-d75bac5cedac admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:57:36.122430 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5eaf66-15da-4aad-9068-d75bac5cedac admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:57:36.123182 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5eaf66-15da-4aad-9068-d75bac5cedac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:57:36.124789 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5eaf66-15da-4aad-9068-d75bac5cedac admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:57:36.134823 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1347/2689] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:57:36 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:57:36.140916 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3cac61-dcf3-47c2-a2f3-f3f426b884ea admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:57:36.157285 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3cac61-dcf3-47c2-a2f3-f3f426b884ea admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:57:36.162687 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ae4b4b-c533-4cc1-97a1-d48960379e22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:57:36.163628 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ae4b4b-c533-4cc1-97a1-d48960379e22 admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:57:36.163916 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ae4b4b-c533-4cc1-97a1-d48960379e22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:57:36.164199 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ae4b4b-c533-4cc1-97a1-d48960379e22 admin admin] PATH_INFO: `/v3/projects/136bb2bdfb31451fb45db10434e04b7f/users/ab87270f385c493faad0c61ea54cffa0/roles/d89b6c92d35a401a8517f1d47f6be843` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:57:36.166211 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ae4b4b-c533-4cc1-97a1-d48960379e22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=136bb2bdfb31451fb45db10434e04b7f, user_id=ab87270f385c493faad0c61ea54cffa0, role_id=d89b6c92d35a401a8517f1d47f6be843)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:57:36.168999 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ae4b4b-c533-4cc1-97a1-d48960379e22 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:57:36.185878 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1343/2690] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:57:36 2022] PUT /identity/v3/projects/136bb2bdfb31451fb45db10434e04b7f/users/ab87270f385c493faad0c61ea54cffa0/roles/d89b6c92d35a401a8517f1d47f6be843 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:57:36.197897 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4a0462-dafa-4202-9e7a-b08e24ea8beb None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:57:36.198191 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4a0462-dafa-4202-9e7a-b08e24ea8beb None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:57:36.198470 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4a0462-dafa-4202-9e7a-b08e24ea8beb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:57:36.222544 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:57:36.225526 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-bc4a0462-dafa-4202-9e7a-b08e24ea8beb None None] MFA Rules not processed for user `ab87270f385c493faad0c61ea54cffa0`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:57:36.226732 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-bc4a0462-dafa-4202-9e7a-b08e24ea8beb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:57:36.295044 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1348/2691] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:57:36 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:57:36.306332 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ae4b4b-c533-4cc1-97a1-d48960379e22 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:57:36.308061 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-47ae4b4b-c533-4cc1-97a1-d48960379e22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:57:36.366372 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ae4b4b-c533-4cc1-97a1-d48960379e22 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:57:36.370094 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34b2f2e-48da-4f1f-9ef7-fd8a72e95e81 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecf4db289fba4e4a806198c62d31f7c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:57:36.370739 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34b2f2e-48da-4f1f-9ef7-fd8a72e95e81 service cinder] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:57:36.371049 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34b2f2e-48da-4f1f-9ef7-fd8a72e95e81 service cinder] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:57:36.371332 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34b2f2e-48da-4f1f-9ef7-fd8a72e95e81 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:57:36.400573 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:57:36.430660 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34b2f2e-48da-4f1f-9ef7-fd8a72e95e81 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:57:36.430863 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34b2f2e-48da-4f1f-9ef7-fd8a72e95e81 service cinder] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:57:36.440099 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1344/2692] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:57:36 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:57:39.497389 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4a0462-dafa-4202-9e7a-b08e24ea8beb None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:57:39.507604 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4a0462-dafa-4202-9e7a-b08e24ea8beb None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:57:39.511422 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04010c08-0faa-4e6f-8930-0fa74e46c7f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecf4db289fba4e4a806198c62d31f7c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:57:39.512103 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04010c08-0faa-4e6f-8930-0fa74e46c7f9 service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:57:39.512392 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04010c08-0faa-4e6f-8930-0fa74e46c7f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:57:39.512674 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04010c08-0faa-4e6f-8930-0fa74e46c7f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:57:39.568161 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04010c08-0faa-4e6f-8930-0fa74e46c7f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:57:39.569600 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04010c08-0faa-4e6f-8930-0fa74e46c7f9 service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:57:39.578965 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1349/2693] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:57:39 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:57:41.308231 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34b2f2e-48da-4f1f-9ef7-fd8a72e95e81 service cinder] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:57:41.316937 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34b2f2e-48da-4f1f-9ef7-fd8a72e95e81 service cinder] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:57:41.320779 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbf1d73-ee34-4dd6-a832-8f5ff2641030 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecf4db289fba4e4a806198c62d31f7c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:57:41.321424 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbf1d73-ee34-4dd6-a832-8f5ff2641030 service cinder] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:57:41.321708 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbf1d73-ee34-4dd6-a832-8f5ff2641030 service cinder] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:57:41.321988 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbf1d73-ee34-4dd6-a832-8f5ff2641030 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:57:41.332347 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbf1d73-ee34-4dd6-a832-8f5ff2641030 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:57:41.333961 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbf1d73-ee34-4dd6-a832-8f5ff2641030 service cinder] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:57:41.343892 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1345/2694] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:57:41 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:57:45.962154 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04010c08-0faa-4e6f-8930-0fa74e46c7f9 service cinder] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:57:46.025645 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04010c08-0faa-4e6f-8930-0fa74e46c7f9 service cinder] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:57:46.030604 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafe6de2-5b47-4452-8510-b34534ce9622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:57:46.031722 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafe6de2-5b47-4452-8510-b34534ce9622 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:57:46.032007 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafe6de2-5b47-4452-8510-b34534ce9622 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:57:46.032286 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafe6de2-5b47-4452-8510-b34534ce9622 admin admin] PATH_INFO: `/v3/users/664d2850bd3b494593fc44851ac01504` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:57:46.036863 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafe6de2-5b47-4452-8510-b34534ce9622 admin admin] RBAC: Authorizing `identity:delete_user(user_id=664d2850bd3b494593fc44851ac01504)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:57:46.036863 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafe6de2-5b47-4452-8510-b34534ce9622 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:57:46.195116 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:57:46.201489 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-fafe6de2-5b47-4452-8510-b34534ce9622 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '664d2850bd3b494593fc44851ac01504'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:57:46.202285 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-fafe6de2-5b47-4452-8510-b34534ce9622 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '664d2850bd3b494593fc44851ac01504'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:57:46.202828 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-fafe6de2-5b47-4452-8510-b34534ce9622 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '664d2850bd3b494593fc44851ac01504'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:57:46.210371 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-fafe6de2-5b47-4452-8510-b34534ce9622 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '664d2850bd3b494593fc44851ac01504'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:57:46.265894 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-fafe6de2-5b47-4452-8510-b34534ce9622 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '664d2850bd3b494593fc44851ac01504'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:57:46.302929 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1350/2695] 10.222.0.81 () {58 vars in 1223 bytes} [Wed Jul 20 22:57:45 2022] DELETE /identity/v3/users/664d2850bd3b494593fc44851ac01504 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:57:46.305112 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babbba77-0731-4d00-8282-90b0552f8b10 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:57:46.305575 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babbba77-0731-4d00-8282-90b0552f8b10 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:57:46.305809 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babbba77-0731-4d00-8282-90b0552f8b10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:57:46.325385 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-babbba77-0731-4d00-8282-90b0552f8b10 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:57:46.326695 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-babbba77-0731-4d00-8282-90b0552f8b10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:57:46.393850 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1346/2696] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:57:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 22:57:46.404339 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafe6de2-5b47-4452-8510-b34534ce9622 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:57:46.406304 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-fafe6de2-5b47-4452-8510-b34534ce9622 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:57:46.486352 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafe6de2-5b47-4452-8510-b34534ce9622 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:57:46.486352 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2e1466-7abe-4e8a-88b1-807e065382df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:57:46.486862 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2e1466-7abe-4e8a-88b1-807e065382df service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:57:46.487184 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2e1466-7abe-4e8a-88b1-807e065382df service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:57:46.487531 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2e1466-7abe-4e8a-88b1-807e065382df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:57:46.498226 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2e1466-7abe-4e8a-88b1-807e065382df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:57:46.499881 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2e1466-7abe-4e8a-88b1-807e065382df service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:57:46.511595 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1351/2697] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:57:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:57:46.911451 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babbba77-0731-4d00-8282-90b0552f8b10 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:57:46.911451 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-babbba77-0731-4d00-8282-90b0552f8b10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:57:46.922542 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babbba77-0731-4d00-8282-90b0552f8b10 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:57:46.928273 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23288ce-c831-4e55-9c15-066177602b2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:57:46.929159 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23288ce-c831-4e55-9c15-066177602b2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:57:46.929504 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23288ce-c831-4e55-9c15-066177602b2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:57:46.929859 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23288ce-c831-4e55-9c15-066177602b2c admin admin] PATH_INFO: `/v3/projects/03fe823060304d7590927c2f6524fac3` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:57:46.931071 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23288ce-c831-4e55-9c15-066177602b2c admin admin] RBAC: Authorizing `identity:delete_project(project_id=03fe823060304d7590927c2f6524fac3)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:57:46.932976 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23288ce-c831-4e55-9c15-066177602b2c admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:57:47.067368 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-e23288ce-c831-4e55-9c15-066177602b2c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03fe823060304d7590927c2f6524fac3', 'request_id': 'req-e23288ce-c831-4e55-9c15-066177602b2c'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:57:47.134608 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-e23288ce-c831-4e55-9c15-066177602b2c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03fe823060304d7590927c2f6524fac3', 'request_id': 'req-e23288ce-c831-4e55-9c15-066177602b2c'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:57:47.141319 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-e23288ce-c831-4e55-9c15-066177602b2c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03fe823060304d7590927c2f6524fac3', 'request_id': 'req-e23288ce-c831-4e55-9c15-066177602b2c'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:57:47.154500 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-e23288ce-c831-4e55-9c15-066177602b2c admin admin] The token cache is being invalidate because project 03fe823060304d7590927c2f6524fac3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:57:47.154796 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-e23288ce-c831-4e55-9c15-066177602b2c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:57:47.155870 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-e23288ce-c831-4e55-9c15-066177602b2c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:57:47.158353 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1347/2698] 10.222.0.81 () {58 vars in 1232 bytes} [Wed Jul 20 22:57:46 2022] DELETE /identity/v3/projects/03fe823060304d7590927c2f6524fac3 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:57:47.162341 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2e1466-7abe-4e8a-88b1-807e065382df service neutron] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:57:47.172507 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2e1466-7abe-4e8a-88b1-807e065382df service neutron] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:57:47.242554 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8644e2c7-c65e-455e-ae2a-c4a9b3241424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:57:47.256226 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8644e2c7-c65e-455e-ae2a-c4a9b3241424 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:57:47.256226 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8644e2c7-c65e-455e-ae2a-c4a9b3241424 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:57:47.256226 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8644e2c7-c65e-455e-ae2a-c4a9b3241424 admin admin] PATH_INFO: `/v3/users/269f6d05b2f2461eb97b398a9b3a40d9` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:57:47.256226 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8644e2c7-c65e-455e-ae2a-c4a9b3241424 admin admin] RBAC: Authorizing `identity:delete_user(user_id=269f6d05b2f2461eb97b398a9b3a40d9)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:57:47.256226 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8644e2c7-c65e-455e-ae2a-c4a9b3241424 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:57:47.452703 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-8644e2c7-c65e-455e-ae2a-c4a9b3241424 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '269f6d05b2f2461eb97b398a9b3a40d9'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:57:47.453368 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-8644e2c7-c65e-455e-ae2a-c4a9b3241424 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '269f6d05b2f2461eb97b398a9b3a40d9'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:57:47.453983 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-8644e2c7-c65e-455e-ae2a-c4a9b3241424 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '269f6d05b2f2461eb97b398a9b3a40d9'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:57:47.461055 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-8644e2c7-c65e-455e-ae2a-c4a9b3241424 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '269f6d05b2f2461eb97b398a9b3a40d9'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:57:47.464693 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:57:47.552096 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-8644e2c7-c65e-455e-ae2a-c4a9b3241424 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '269f6d05b2f2461eb97b398a9b3a40d9'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:57:47.571229 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1352/2699] 10.222.0.81 () {58 vars in 1223 bytes} [Wed Jul 20 22:57:47 2022] DELETE /identity/v3/users/269f6d05b2f2461eb97b398a9b3a40d9 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:57:47.987452 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23288ce-c831-4e55-9c15-066177602b2c admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:57:47.988966 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-e23288ce-c831-4e55-9c15-066177602b2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:57:48.003881 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23288ce-c831-4e55-9c15-066177602b2c admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:57:48.040588 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:57:48.063209 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18d58f8-66c4-424b-8336-888bb1b0906b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:57:48.064102 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18d58f8-66c4-424b-8336-888bb1b0906b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:57:48.064414 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18d58f8-66c4-424b-8336-888bb1b0906b admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:57:48.064695 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18d58f8-66c4-424b-8336-888bb1b0906b admin admin] PATH_INFO: `/v3/projects/d71193306ded43beb8136d938b52d2cb` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:57:48.065912 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18d58f8-66c4-424b-8336-888bb1b0906b admin admin] RBAC: Authorizing `identity:delete_project(project_id=d71193306ded43beb8136d938b52d2cb)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:57:48.067861 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18d58f8-66c4-424b-8336-888bb1b0906b admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:57:48.202985 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-e18d58f8-66c4-424b-8336-888bb1b0906b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd71193306ded43beb8136d938b52d2cb', 'request_id': 'req-e18d58f8-66c4-424b-8336-888bb1b0906b'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:57:48.290153 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-e18d58f8-66c4-424b-8336-888bb1b0906b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd71193306ded43beb8136d938b52d2cb', 'request_id': 'req-e18d58f8-66c4-424b-8336-888bb1b0906b'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:57:48.298832 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-e18d58f8-66c4-424b-8336-888bb1b0906b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd71193306ded43beb8136d938b52d2cb', 'request_id': 'req-e18d58f8-66c4-424b-8336-888bb1b0906b'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:57:48.310322 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-e18d58f8-66c4-424b-8336-888bb1b0906b admin admin] The token cache is being invalidate because project d71193306ded43beb8136d938b52d2cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:57:48.310616 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-e18d58f8-66c4-424b-8336-888bb1b0906b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:57:48.311322 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-e18d58f8-66c4-424b-8336-888bb1b0906b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:57:48.312868 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1348/2700] 10.222.0.81 () {58 vars in 1232 bytes} [Wed Jul 20 22:57:47 2022] DELETE /identity/v3/projects/d71193306ded43beb8136d938b52d2cb => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:57:48.316891 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8644e2c7-c65e-455e-ae2a-c4a9b3241424 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:57:48.318333 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-8644e2c7-c65e-455e-ae2a-c4a9b3241424 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:57:48.331090 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8644e2c7-c65e-455e-ae2a-c4a9b3241424 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:57:48.389346 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e03dd44-5a6f-4162-b996-68ebde04f212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:57:48.390346 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e03dd44-5a6f-4162-b996-68ebde04f212 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:57:48.390962 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e03dd44-5a6f-4162-b996-68ebde04f212 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:57:48.390962 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e03dd44-5a6f-4162-b996-68ebde04f212 admin admin] PATH_INFO: `/v3/users/ab87270f385c493faad0c61ea54cffa0` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:57:48.401695 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e03dd44-5a6f-4162-b996-68ebde04f212 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab87270f385c493faad0c61ea54cffa0)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:57:48.401695 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e03dd44-5a6f-4162-b996-68ebde04f212 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:57:48.565489 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-9e03dd44-5a6f-4162-b996-68ebde04f212 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab87270f385c493faad0c61ea54cffa0'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:57:48.566153 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-9e03dd44-5a6f-4162-b996-68ebde04f212 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab87270f385c493faad0c61ea54cffa0'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:57:48.566759 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-9e03dd44-5a6f-4162-b996-68ebde04f212 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab87270f385c493faad0c61ea54cffa0'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:57:48.574127 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-9e03dd44-5a6f-4162-b996-68ebde04f212 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab87270f385c493faad0c61ea54cffa0'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:57:48.654224 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-9e03dd44-5a6f-4162-b996-68ebde04f212 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab87270f385c493faad0c61ea54cffa0'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:57:48.672637 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1353/2701] 10.222.0.81 () {58 vars in 1223 bytes} [Wed Jul 20 22:57:48 2022] DELETE /identity/v3/users/ab87270f385c493faad0c61ea54cffa0 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:57:49.206201 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18d58f8-66c4-424b-8336-888bb1b0906b admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:57:49.207813 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-e18d58f8-66c4-424b-8336-888bb1b0906b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:57:49.222060 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18d58f8-66c4-424b-8336-888bb1b0906b admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:57:49.285886 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d10592-30bd-4f8c-a86c-f8723366f285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:57:49.286778 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d10592-30bd-4f8c-a86c-f8723366f285 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:57:49.287099 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d10592-30bd-4f8c-a86c-f8723366f285 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:57:49.287381 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d10592-30bd-4f8c-a86c-f8723366f285 admin admin] PATH_INFO: `/v3/projects/136bb2bdfb31451fb45db10434e04b7f` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:57:49.288579 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d10592-30bd-4f8c-a86c-f8723366f285 admin admin] RBAC: Authorizing `identity:delete_project(project_id=136bb2bdfb31451fb45db10434e04b7f)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:57:49.290506 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d10592-30bd-4f8c-a86c-f8723366f285 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:57:49.558573 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-61d10592-30bd-4f8c-a86c-f8723366f285 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '136bb2bdfb31451fb45db10434e04b7f', 'request_id': 'req-61d10592-30bd-4f8c-a86c-f8723366f285'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:57:49.717207 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-61d10592-30bd-4f8c-a86c-f8723366f285 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '136bb2bdfb31451fb45db10434e04b7f', 'request_id': 'req-61d10592-30bd-4f8c-a86c-f8723366f285'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:57:49.723970 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-61d10592-30bd-4f8c-a86c-f8723366f285 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '136bb2bdfb31451fb45db10434e04b7f', 'request_id': 'req-61d10592-30bd-4f8c-a86c-f8723366f285'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:57:49.731181 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:57:49.735569 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-61d10592-30bd-4f8c-a86c-f8723366f285 admin admin] The token cache is being invalidate because project 136bb2bdfb31451fb45db10434e04b7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:57:49.736396 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-61d10592-30bd-4f8c-a86c-f8723366f285 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:57:49.737064 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-61d10592-30bd-4f8c-a86c-f8723366f285 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:57:49.738640 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1349/2702] 10.222.0.81 () {58 vars in 1232 bytes} [Wed Jul 20 22:57:49 2022] DELETE /identity/v3/projects/136bb2bdfb31451fb45db10434e04b7f => generated 0 bytes in 534 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:57:49.743587 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ea268d-29bb-4962-a2c9-db6bf7a36374 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:57:49.744074 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ea268d-29bb-4962-a2c9-db6bf7a36374 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:57:49.744183 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ea268d-29bb-4962-a2c9-db6bf7a36374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:57:49.765677 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-f6ea268d-29bb-4962-a2c9-db6bf7a36374 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:57:49.766946 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-f6ea268d-29bb-4962-a2c9-db6bf7a36374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:57:49.821315 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:57:49.831710 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1354/2703] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:57:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:57:49.840036 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ec5012-a4ec-46b0-b4ba-4476bf36e44e None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:57:49.840337 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ec5012-a4ec-46b0-b4ba-4476bf36e44e None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:57:49.840624 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ec5012-a4ec-46b0-b4ba-4476bf36e44e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:57:49.859174 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-c7ec5012-a4ec-46b0-b4ba-4476bf36e44e None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:57:49.860387 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-c7ec5012-a4ec-46b0-b4ba-4476bf36e44e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:57:49.868896 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1350/2704] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:57:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:57:49.873116 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ea268d-29bb-4962-a2c9-db6bf7a36374 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:57:49.883141 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ea268d-29bb-4962-a2c9-db6bf7a36374 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:57:49.888225 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb83bbe9-284d-495a-82d6-b267d8e3f9b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:57:49.889895 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb83bbe9-284d-495a-82d6-b267d8e3f9b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:57:49.889895 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb83bbe9-284d-495a-82d6-b267d8e3f9b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:57:49.889895 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb83bbe9-284d-495a-82d6-b267d8e3f9b0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:57:49.890137 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb83bbe9-284d-495a-82d6-b267d8e3f9b0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:57:49.890470 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb83bbe9-284d-495a-82d6-b267d8e3f9b0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:57:49.892234 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb83bbe9-284d-495a-82d6-b267d8e3f9b0 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:57:49.908408 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1355/2705] 10.222.0.81 () {58 vars in 1152 bytes} [Wed Jul 20 22:57:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:57:49.910082 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ec5012-a4ec-46b0-b4ba-4476bf36e44e None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:57:49.920452 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ec5012-a4ec-46b0-b4ba-4476bf36e44e None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:57:49.926455 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d7a1f7-bfa0-4192-a6ab-12a8267bc8c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:57:49.929952 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d7a1f7-bfa0-4192-a6ab-12a8267bc8c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:57:49.929952 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d7a1f7-bfa0-4192-a6ab-12a8267bc8c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:57:49.929952 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d7a1f7-bfa0-4192-a6ab-12a8267bc8c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:57:49.929952 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d7a1f7-bfa0-4192-a6ab-12a8267bc8c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:57:49.930353 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d7a1f7-bfa0-4192-a6ab-12a8267bc8c7 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:57:50.051447 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1351/2706] 10.222.0.81 () {60 vars in 1152 bytes} [Wed Jul 20 22:57:49 2022] POST /identity/v3/projects => generated 373 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:57:50.055722 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb83bbe9-284d-495a-82d6-b267d8e3f9b0 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:57:50.065760 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb83bbe9-284d-495a-82d6-b267d8e3f9b0 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:57:50.124974 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a2cf6d-55d9-4b51-8c8a-8a79d351ad15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:57:50.125660 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a2cf6d-55d9-4b51-8c8a-8a79d351ad15 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:57:50.125948 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a2cf6d-55d9-4b51-8c8a-8a79d351ad15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:57:50.126225 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a2cf6d-55d9-4b51-8c8a-8a79d351ad15 admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:57:50.126929 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a2cf6d-55d9-4b51-8c8a-8a79d351ad15 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:57:50.128853 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a2cf6d-55d9-4b51-8c8a-8a79d351ad15 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:57:50.204941 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1356/2707] 10.222.0.81 () {60 vars in 1143 bytes} [Wed Jul 20 22:57:50 2022] POST /identity/v3/users => generated 287 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:57:50.209113 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d7a1f7-bfa0-4192-a6ab-12a8267bc8c7 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:57:50.221456 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d7a1f7-bfa0-4192-a6ab-12a8267bc8c7 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:57:50.226272 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7836fa6a-b4da-4ef2-bd16-f126fc679a16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:57:50.226950 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7836fa6a-b4da-4ef2-bd16-f126fc679a16 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:57:50.227282 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7836fa6a-b4da-4ef2-bd16-f126fc679a16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:57:50.227570 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7836fa6a-b4da-4ef2-bd16-f126fc679a16 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:57:50.228314 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7836fa6a-b4da-4ef2-bd16-f126fc679a16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:57:50.230050 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7836fa6a-b4da-4ef2-bd16-f126fc679a16 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:57:50.240728 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1352/2708] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:57:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:57:50.244696 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a2cf6d-55d9-4b51-8c8a-8a79d351ad15 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:57:50.254803 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a2cf6d-55d9-4b51-8c8a-8a79d351ad15 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:57:50.259598 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4ef7c3-f50e-4de3-857a-747755787383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:57:50.260518 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4ef7c3-f50e-4de3-857a-747755787383 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:57:50.260806 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4ef7c3-f50e-4de3-857a-747755787383 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:57:50.261086 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4ef7c3-f50e-4de3-857a-747755787383 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:57:50.261833 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4ef7c3-f50e-4de3-857a-747755787383 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:57:50.263425 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4ef7c3-f50e-4de3-857a-747755787383 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:57:50.274018 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1357/2709] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:57:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:57:50.278020 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7836fa6a-b4da-4ef2-bd16-f126fc679a16 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:57:50.289437 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7836fa6a-b4da-4ef2-bd16-f126fc679a16 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:57:50.294204 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08402f5-8087-43de-81cb-f35940963c94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:57:50.295133 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08402f5-8087-43de-81cb-f35940963c94 admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:57:50.295422 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08402f5-8087-43de-81cb-f35940963c94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:57:50.295701 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08402f5-8087-43de-81cb-f35940963c94 admin admin] PATH_INFO: `/v3/projects/2052ef42be464cc98e98aa5da24ee7ae/users/d8e921b8bae145aeb67fe1f1497c502e/roles/559942c4de1744169f8b5e0404ecde6b` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:57:50.310369 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08402f5-8087-43de-81cb-f35940963c94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2052ef42be464cc98e98aa5da24ee7ae, user_id=d8e921b8bae145aeb67fe1f1497c502e, role_id=559942c4de1744169f8b5e0404ecde6b)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:57:50.313075 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08402f5-8087-43de-81cb-f35940963c94 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:57:50.403324 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1353/2710] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:57:50 2022] PUT /identity/v3/projects/2052ef42be464cc98e98aa5da24ee7ae/users/d8e921b8bae145aeb67fe1f1497c502e/roles/559942c4de1744169f8b5e0404ecde6b => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:57:50.408483 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f0ccf3-b198-4699-9a2e-9461382f9885 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:57:50.408780 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f0ccf3-b198-4699-9a2e-9461382f9885 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:57:50.409057 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f0ccf3-b198-4699-9a2e-9461382f9885 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:57:50.427149 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-f3f0ccf3-b198-4699-9a2e-9461382f9885 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:57:50.433372 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-f3f0ccf3-b198-4699-9a2e-9461382f9885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:57:50.489404 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1358/2711] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:57:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:57:50.499355 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08402f5-8087-43de-81cb-f35940963c94 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:57:50.501067 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-a08402f5-8087-43de-81cb-f35940963c94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:57:50.559720 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08402f5-8087-43de-81cb-f35940963c94 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:57:50.563442 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b6839b-d237-4038-bc0a-32ff16b7eeec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:57:50.564096 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b6839b-d237-4038-bc0a-32ff16b7eeec service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:57:50.564382 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b6839b-d237-4038-bc0a-32ff16b7eeec service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:57:50.564659 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b6839b-d237-4038-bc0a-32ff16b7eeec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:57:50.575116 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b6839b-d237-4038-bc0a-32ff16b7eeec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:57:50.576565 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b6839b-d237-4038-bc0a-32ff16b7eeec service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:57:50.588611 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1354/2712] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:57:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:57:51.850723 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f1dca9-af28-4fb1-ad38-de1741ee8f45 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:57:51.851317 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f1dca9-af28-4fb1-ad38-de1741ee8f45 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:57:51.851317 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f1dca9-af28-4fb1-ad38-de1741ee8f45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:57:51.868861 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-52f1dca9-af28-4fb1-ad38-de1741ee8f45 None None] MFA Rules not processed for user `bc19753d420840ffa5cde19d4c1fac8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:57:51.870062 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-52f1dca9-af28-4fb1-ad38-de1741ee8f45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:57:51.940202 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1359/2713] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:57:51 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:57:52.115488 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966731cb-38b5-40db-ad53-c6326016da97 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:57:52.120274 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966731cb-38b5-40db-ad53-c6326016da97 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:57:52.120274 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966731cb-38b5-40db-ad53-c6326016da97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:57:52.146111 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-966731cb-38b5-40db-ad53-c6326016da97 None None] MFA Rules not processed for user `d8e921b8bae145aeb67fe1f1497c502e`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:57:52.148174 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-966731cb-38b5-40db-ad53-c6326016da97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:57:52.231863 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1355/2714] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:57:52 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:57:52.240268 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f1dca9-af28-4fb1-ad38-de1741ee8f45 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:57:52.241709 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-52f1dca9-af28-4fb1-ad38-de1741ee8f45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:57:52.252979 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:57:52.300796 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966731cb-38b5-40db-ad53-c6326016da97 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:57:52.301294 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f1dca9-af28-4fb1-ad38-de1741ee8f45 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:57:52.302524 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-966731cb-38b5-40db-ad53-c6326016da97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:57:52.306771 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42e47bf-c4d0-4c6f-9d20-683bc3b973e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecf4db289fba4e4a806198c62d31f7c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:57:52.309591 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42e47bf-c4d0-4c6f-9d20-683bc3b973e5 service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:57:52.309591 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42e47bf-c4d0-4c6f-9d20-683bc3b973e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:57:52.309591 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42e47bf-c4d0-4c6f-9d20-683bc3b973e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:57:52.315797 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966731cb-38b5-40db-ad53-c6326016da97 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:57:52.319883 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42e47bf-c4d0-4c6f-9d20-683bc3b973e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:57:52.320056 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc65ae3-5b4e-4a83-bb99-57e08b5026b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:57:52.320713 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc65ae3-5b4e-4a83-bb99-57e08b5026b5 service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:57:52.321056 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc65ae3-5b4e-4a83-bb99-57e08b5026b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:57:52.321288 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc65ae3-5b4e-4a83-bb99-57e08b5026b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:57:52.321441 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42e47bf-c4d0-4c6f-9d20-683bc3b973e5 service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:57:52.323135 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-8fc65ae3-5b4e-4a83-bb99-57e08b5026b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:57:52.331111 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1360/2715] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:57:52 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:57:52.338182 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc65ae3-5b4e-4a83-bb99-57e08b5026b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:57:52.338182 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc65ae3-5b4e-4a83-bb99-57e08b5026b5 service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:57:52.344348 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1356/2716] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:57:52 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 22:57:55.197014 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e847d25-1634-4cda-8997-2453b6fecf11 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:57:55.197411 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e847d25-1634-4cda-8997-2453b6fecf11 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:57:55.197728 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e847d25-1634-4cda-8997-2453b6fecf11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:57:55.219122 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-9e847d25-1634-4cda-8997-2453b6fecf11 None None] MFA Rules not processed for user `d8e921b8bae145aeb67fe1f1497c502e`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:57:55.220715 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-9e847d25-1634-4cda-8997-2453b6fecf11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:57:55.228100 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1361/2717] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:57:55 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:58:02.341338 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=92,keystone:INSERT=2 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:58:04.879890 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc65ae3-5b4e-4a83-bb99-57e08b5026b5 service neutron] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:58:04.881307 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-8fc65ae3-5b4e-4a83-bb99-57e08b5026b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:58:04.958587 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc65ae3-5b4e-4a83-bb99-57e08b5026b5 service neutron] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:58:04.964699 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1503cb8-a94b-4034-aa3e-37758218137f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7829324db2574d38b18c749d0559177f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:58:04.965362 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1503cb8-a94b-4034-aa3e-37758218137f service nova] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:58:04.965676 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1503cb8-a94b-4034-aa3e-37758218137f service nova] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:58:04.965986 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1503cb8-a94b-4034-aa3e-37758218137f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:58:04.968054 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-e1503cb8-a94b-4034-aa3e-37758218137f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:58:04.982255 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1503cb8-a94b-4034-aa3e-37758218137f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:58:04.984810 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1503cb8-a94b-4034-aa3e-37758218137f service nova] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:58:04.995103 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1357/2718] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:58:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:58:05.213806 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=25 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:58:14.990467 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=34 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:58:17.010711 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20cf73a-bab3-4264-b822-b900145ec668 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:58:17.011253 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20cf73a-bab3-4264-b822-b900145ec668 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:58:17.011376 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20cf73a-bab3-4264-b822-b900145ec668 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:58:17.029141 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-a20cf73a-bab3-4264-b822-b900145ec668 None None] MFA Rules not processed for user `d8e921b8bae145aeb67fe1f1497c502e`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:58:17.030352 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-a20cf73a-bab3-4264-b822-b900145ec668 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:58:17.037821 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1362/2719] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:58:17 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 22:58:17.268179 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1503cb8-a94b-4034-aa3e-37758218137f service nova] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:58:17.277475 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1503cb8-a94b-4034-aa3e-37758218137f service nova] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:58:17.281044 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f24a2e-d4dd-4aa1-8cd6-3edf08807f83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:58:17.281694 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f24a2e-d4dd-4aa1-8cd6-3edf08807f83 service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:58:17.281979 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f24a2e-d4dd-4aa1-8cd6-3edf08807f83 service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:58:17.282253 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f24a2e-d4dd-4aa1-8cd6-3edf08807f83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:58:17.284071 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-67f24a2e-d4dd-4aa1-8cd6-3edf08807f83 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:58:17.293289 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f24a2e-d4dd-4aa1-8cd6-3edf08807f83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:58:17.294703 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f24a2e-d4dd-4aa1-8cd6-3edf08807f83 service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:58:17.303087 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1358/2720] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:58:17 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:58:25.121787 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20cf73a-bab3-4264-b822-b900145ec668 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:58:25.131375 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20cf73a-bab3-4264-b822-b900145ec668 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:58:25.134913 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c32d91f-0d8c-4a37-ad99-1958e6ff0a4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:58:25.135843 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c32d91f-0d8c-4a37-ad99-1958e6ff0a4e service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:58:25.136128 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c32d91f-0d8c-4a37-ad99-1958e6ff0a4e service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:58:25.136407 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c32d91f-0d8c-4a37-ad99-1958e6ff0a4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:58:25.138163 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-9c32d91f-0d8c-4a37-ad99-1958e6ff0a4e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:58:25.196549 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c32d91f-0d8c-4a37-ad99-1958e6ff0a4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:58:25.197985 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c32d91f-0d8c-4a37-ad99-1958e6ff0a4e service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:58:25.206962 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1363/2721] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:58:25 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:58:27.299083 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=6 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:58:32.795125 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f24a2e-d4dd-4aa1-8cd6-3edf08807f83 service neutron] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:58:32.804545 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f24a2e-d4dd-4aa1-8cd6-3edf08807f83 service neutron] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:58:32.808614 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e768f7-d9bc-4985-af42-dba9dc802c03 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:58:32.809450 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e768f7-d9bc-4985-af42-dba9dc802c03 service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:58:32.809926 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e768f7-d9bc-4985-af42-dba9dc802c03 service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:58:32.810215 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e768f7-d9bc-4985-af42-dba9dc802c03 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:58:32.812051 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-17e768f7-d9bc-4985-af42-dba9dc802c03 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:58:32.826198 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e768f7-d9bc-4985-af42-dba9dc802c03 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:58:32.827674 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e768f7-d9bc-4985-af42-dba9dc802c03 service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:58:32.837927 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1359/2722] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:58:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:58:35.202828 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=31 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:58:37.057052 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c32d91f-0d8c-4a37-ad99-1958e6ff0a4e service neutron] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:58:37.058497 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-9c32d91f-0d8c-4a37-ad99-1958e6ff0a4e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:58:37.130057 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c32d91f-0d8c-4a37-ad99-1958e6ff0a4e service neutron] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:58:37.135616 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee24e7f-c935-4254-96cb-fbb762965f75 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5500cebdd9d4f8e9aecb45d6c955878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:58:37.136277 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee24e7f-c935-4254-96cb-fbb762965f75 service placement] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:58:37.141942 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee24e7f-c935-4254-96cb-fbb762965f75 service placement] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:58:37.141942 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee24e7f-c935-4254-96cb-fbb762965f75 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:58:37.141942 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-3ee24e7f-c935-4254-96cb-fbb762965f75 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:58:37.150682 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee24e7f-c935-4254-96cb-fbb762965f75 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:58:37.152303 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee24e7f-c935-4254-96cb-fbb762965f75 service placement] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:58:37.162811 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1364/2723] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:58:37 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:58:40.769134 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e768f7-d9bc-4985-af42-dba9dc802c03 service neutron] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:58:40.771269 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-17e768f7-d9bc-4985-af42-dba9dc802c03 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:58:40.784149 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e768f7-d9bc-4985-af42-dba9dc802c03 service neutron] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:58:40.788875 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707331a3-30fd-4754-a3dc-7985c12b451e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:58:40.808819 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707331a3-30fd-4754-a3dc-7985c12b451e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:58:40.808819 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707331a3-30fd-4754-a3dc-7985c12b451e admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:58:40.808819 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707331a3-30fd-4754-a3dc-7985c12b451e admin admin] PATH_INFO: `/v3/users/3ce4c02d3e314761ba613a802194b709` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:58:40.808819 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707331a3-30fd-4754-a3dc-7985c12b451e admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ce4c02d3e314761ba613a802194b709)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:58:40.808819 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707331a3-30fd-4754-a3dc-7985c12b451e admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:58:40.970461 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-707331a3-30fd-4754-a3dc-7985c12b451e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ce4c02d3e314761ba613a802194b709'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:58:40.971145 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-707331a3-30fd-4754-a3dc-7985c12b451e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ce4c02d3e314761ba613a802194b709'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:58:40.971761 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-707331a3-30fd-4754-a3dc-7985c12b451e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ce4c02d3e314761ba613a802194b709'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:58:40.978785 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-707331a3-30fd-4754-a3dc-7985c12b451e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ce4c02d3e314761ba613a802194b709'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:58:41.030839 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-707331a3-30fd-4754-a3dc-7985c12b451e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ce4c02d3e314761ba613a802194b709'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:58:41.048251 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1360/2724] 10.222.0.81 () {58 vars in 1223 bytes} [Wed Jul 20 22:58:40 2022] DELETE /identity/v3/users/3ce4c02d3e314761ba613a802194b709 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:58:41.546550 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee24e7f-c935-4254-96cb-fbb762965f75 service placement] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:58:41.557457 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee24e7f-c935-4254-96cb-fbb762965f75 service placement] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:58:41.613719 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c363cb-d466-4b30-baeb-3cbc7c7a9780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:58:41.614604 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c363cb-d466-4b30-baeb-3cbc7c7a9780 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:58:41.614888 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c363cb-d466-4b30-baeb-3cbc7c7a9780 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:58:41.615195 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c363cb-d466-4b30-baeb-3cbc7c7a9780 admin admin] PATH_INFO: `/v3/projects/a257ecd598a74a5cb15fc4b62c5a5979` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:58:41.616378 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c363cb-d466-4b30-baeb-3cbc7c7a9780 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a257ecd598a74a5cb15fc4b62c5a5979)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:58:41.618265 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c363cb-d466-4b30-baeb-3cbc7c7a9780 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:58:41.722608 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-d2c363cb-d466-4b30-baeb-3cbc7c7a9780 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a257ecd598a74a5cb15fc4b62c5a5979', 'request_id': 'req-d2c363cb-d466-4b30-baeb-3cbc7c7a9780'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:58:41.728691 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-d2c363cb-d466-4b30-baeb-3cbc7c7a9780 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a257ecd598a74a5cb15fc4b62c5a5979', 'request_id': 'req-d2c363cb-d466-4b30-baeb-3cbc7c7a9780'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:58:41.808231 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-d2c363cb-d466-4b30-baeb-3cbc7c7a9780 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a257ecd598a74a5cb15fc4b62c5a5979', 'request_id': 'req-d2c363cb-d466-4b30-baeb-3cbc7c7a9780'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:58:41.819650 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-d2c363cb-d466-4b30-baeb-3cbc7c7a9780 admin admin] The token cache is being invalidate because project a257ecd598a74a5cb15fc4b62c5a5979 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:58:41.823802 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-d2c363cb-d466-4b30-baeb-3cbc7c7a9780 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:58:41.823802 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-d2c363cb-d466-4b30-baeb-3cbc7c7a9780 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:58:41.823802 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1365/2725] 10.222.0.81 () {58 vars in 1232 bytes} [Wed Jul 20 22:58:41 2022] DELETE /identity/v3/projects/a257ecd598a74a5cb15fc4b62c5a5979 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:58:41.826182 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707331a3-30fd-4754-a3dc-7985c12b451e admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:58:41.827638 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-707331a3-30fd-4754-a3dc-7985c12b451e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:58:41.840615 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707331a3-30fd-4754-a3dc-7985c12b451e admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:58:41.897030 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbcdcd7-8548-4181-b3ed-307fcec82d2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:58:41.897925 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbcdcd7-8548-4181-b3ed-307fcec82d2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:58:41.898213 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbcdcd7-8548-4181-b3ed-307fcec82d2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:58:41.898496 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbcdcd7-8548-4181-b3ed-307fcec82d2a admin admin] PATH_INFO: `/v3/users/020bbaed1f9148c4a9daa75180935b8e` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:58:41.899729 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbcdcd7-8548-4181-b3ed-307fcec82d2a admin admin] RBAC: Authorizing `identity:delete_user(user_id=020bbaed1f9148c4a9daa75180935b8e)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:58:41.901651 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbcdcd7-8548-4181-b3ed-307fcec82d2a admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:58:42.104346 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-bdbcdcd7-8548-4181-b3ed-307fcec82d2a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '020bbaed1f9148c4a9daa75180935b8e'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:58:42.104995 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-bdbcdcd7-8548-4181-b3ed-307fcec82d2a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '020bbaed1f9148c4a9daa75180935b8e'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:58:42.105612 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-bdbcdcd7-8548-4181-b3ed-307fcec82d2a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '020bbaed1f9148c4a9daa75180935b8e'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:58:42.112570 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-bdbcdcd7-8548-4181-b3ed-307fcec82d2a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '020bbaed1f9148c4a9daa75180935b8e'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:58:42.116430 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:58:42.198812 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-bdbcdcd7-8548-4181-b3ed-307fcec82d2a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '020bbaed1f9148c4a9daa75180935b8e'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:58:42.221935 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1361/2726] 10.222.0.81 () {58 vars in 1223 bytes} [Wed Jul 20 22:58:41 2022] DELETE /identity/v3/users/020bbaed1f9148c4a9daa75180935b8e => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:58:42.478050 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c363cb-d466-4b30-baeb-3cbc7c7a9780 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:58:42.479512 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-d2c363cb-d466-4b30-baeb-3cbc7c7a9780 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:58:42.493156 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c363cb-d466-4b30-baeb-3cbc7c7a9780 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:58:42.511650 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:58:42.550201 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f08069-5742-47e3-b77a-2c8affd36a4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:58:42.551604 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f08069-5742-47e3-b77a-2c8affd36a4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:58:42.551893 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f08069-5742-47e3-b77a-2c8affd36a4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:58:42.552170 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f08069-5742-47e3-b77a-2c8affd36a4c admin admin] PATH_INFO: `/v3/projects/0e9874be0cd542829242caded150fd52` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:58:42.553351 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f08069-5742-47e3-b77a-2c8affd36a4c admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e9874be0cd542829242caded150fd52)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:58:42.555281 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f08069-5742-47e3-b77a-2c8affd36a4c admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:58:42.643604 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-23f08069-5742-47e3-b77a-2c8affd36a4c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e9874be0cd542829242caded150fd52', 'request_id': 'req-23f08069-5742-47e3-b77a-2c8affd36a4c'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:58:42.649708 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-23f08069-5742-47e3-b77a-2c8affd36a4c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e9874be0cd542829242caded150fd52', 'request_id': 'req-23f08069-5742-47e3-b77a-2c8affd36a4c'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:58:42.705626 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-23f08069-5742-47e3-b77a-2c8affd36a4c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e9874be0cd542829242caded150fd52', 'request_id': 'req-23f08069-5742-47e3-b77a-2c8affd36a4c'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:58:42.722761 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-23f08069-5742-47e3-b77a-2c8affd36a4c admin admin] The token cache is being invalidate because project 0e9874be0cd542829242caded150fd52 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:58:42.722979 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-23f08069-5742-47e3-b77a-2c8affd36a4c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:58:42.723712 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-23f08069-5742-47e3-b77a-2c8affd36a4c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:58:42.725292 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1366/2727] 10.222.0.81 () {58 vars in 1232 bytes} [Wed Jul 20 22:58:42 2022] DELETE /identity/v3/projects/0e9874be0cd542829242caded150fd52 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:58:44.064584 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e9d4ec-c9e9-4b83-ae5a-9906a3c98c50 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:58:44.065628 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e9d4ec-c9e9-4b83-ae5a-9906a3c98c50 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:58:44.065628 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e9d4ec-c9e9-4b83-ae5a-9906a3c98c50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:58:44.083354 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-c9e9d4ec-c9e9-4b83-ae5a-9906a3c98c50 None None] MFA Rules not processed for user `90915e1264324f16bc7bae7e46e0c97b`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:58:44.084552 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-c9e9d4ec-c9e9-4b83-ae5a-9906a3c98c50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:58:44.143321 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1362/2728] 10.222.0.81 () {58 vars in 1009 bytes} [Wed Jul 20 22:58:44 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:58:44.258526 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f08069-5742-47e3-b77a-2c8affd36a4c admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:58:44.260033 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-23f08069-5742-47e3-b77a-2c8affd36a4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:58:44.329422 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f08069-5742-47e3-b77a-2c8affd36a4c admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:58:44.334447 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98cbde0-b08f-40cc-920d-283da2900eef service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70e97436a08543a180225561ef8bc817', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:58:44.335120 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98cbde0-b08f-40cc-920d-283da2900eef service swift] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:58:44.335410 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98cbde0-b08f-40cc-920d-283da2900eef service swift] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:58:44.335686 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98cbde0-b08f-40cc-920d-283da2900eef service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:58:44.343561 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98cbde0-b08f-40cc-920d-283da2900eef service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:58:44.345126 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98cbde0-b08f-40cc-920d-283da2900eef service swift] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:58:44.353553 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1367/2729] 10.222.0.81 () {60 vars in 1384 bytes} [Wed Jul 20 22:58:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 22:58:54.136752 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=33,keystone:DELETE=2 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:58:54.350357 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=74,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:59:00.725507 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976f4acb-75f4-4449-8d57-d7a3594dd13f None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:59:00.726060 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976f4acb-75f4-4449-8d57-d7a3594dd13f None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:59:00.726060 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976f4acb-75f4-4449-8d57-d7a3594dd13f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:59:00.744684 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-976f4acb-75f4-4449-8d57-d7a3594dd13f None None] MFA Rules not processed for user `90915e1264324f16bc7bae7e46e0c97b`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:59:00.746587 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-976f4acb-75f4-4449-8d57-d7a3594dd13f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:59:00.753434 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1363/2730] 10.222.0.81 () {58 vars in 967 bytes} [Wed Jul 20 22:59:00 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:59:00.767577 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98cbde0-b08f-40cc-920d-283da2900eef service swift] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:59:00.779196 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98cbde0-b08f-40cc-920d-283da2900eef service swift] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:59:00.786767 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c15fd6-a2da-420b-9f4a-2ba69f02cf19 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70e97436a08543a180225561ef8bc817', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:59:00.786767 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c15fd6-a2da-420b-9f4a-2ba69f02cf19 service swift] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:59:00.786767 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c15fd6-a2da-420b-9f4a-2ba69f02cf19 service swift] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:59:00.786767 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c15fd6-a2da-420b-9f4a-2ba69f02cf19 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:59:00.796021 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c15fd6-a2da-420b-9f4a-2ba69f02cf19 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:59:00.797649 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c15fd6-a2da-420b-9f4a-2ba69f02cf19 service swift] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:59:00.806939 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1368/2731] 10.222.0.81 () {60 vars in 1384 bytes} [Wed Jul 20 22:59:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 22:59:04.100721 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976f4acb-75f4-4449-8d57-d7a3594dd13f None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:59:04.102191 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-976f4acb-75f4-4449-8d57-d7a3594dd13f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:59:04.168095 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976f4acb-75f4-4449-8d57-d7a3594dd13f None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:59:04.172734 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37b289d-3949-4dff-b7e3-4ab40c9a7c13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:59:04.173606 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37b289d-3949-4dff-b7e3-4ab40c9a7c13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:59:04.173884 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37b289d-3949-4dff-b7e3-4ab40c9a7c13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:59:04.174183 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37b289d-3949-4dff-b7e3-4ab40c9a7c13 admin admin] PATH_INFO: `/v3/users/d8e921b8bae145aeb67fe1f1497c502e` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:59:04.175401 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37b289d-3949-4dff-b7e3-4ab40c9a7c13 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8e921b8bae145aeb67fe1f1497c502e)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:59:04.178039 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37b289d-3949-4dff-b7e3-4ab40c9a7c13 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:59:04.306580 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-d37b289d-3949-4dff-b7e3-4ab40c9a7c13 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8e921b8bae145aeb67fe1f1497c502e'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:59:04.307272 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-d37b289d-3949-4dff-b7e3-4ab40c9a7c13 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8e921b8bae145aeb67fe1f1497c502e'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:59:04.307868 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-d37b289d-3949-4dff-b7e3-4ab40c9a7c13 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8e921b8bae145aeb67fe1f1497c502e'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:59:04.314706 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-d37b289d-3949-4dff-b7e3-4ab40c9a7c13 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8e921b8bae145aeb67fe1f1497c502e'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:59:04.362368 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-d37b289d-3949-4dff-b7e3-4ab40c9a7c13 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8e921b8bae145aeb67fe1f1497c502e'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:59:04.381539 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1364/2732] 10.222.0.81 () {58 vars in 1223 bytes} [Wed Jul 20 22:59:04 2022] DELETE /identity/v3/users/d8e921b8bae145aeb67fe1f1497c502e => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:59:04.630000 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c15fd6-a2da-420b-9f4a-2ba69f02cf19 service swift] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:59:04.642190 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c15fd6-a2da-420b-9f4a-2ba69f02cf19 service swift] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:59:04.698476 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf389b17-075d-4e5d-9c0e-a061bb5bfbb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:59:04.699393 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf389b17-075d-4e5d-9c0e-a061bb5bfbb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:59:04.699679 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf389b17-075d-4e5d-9c0e-a061bb5bfbb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:59:04.699957 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf389b17-075d-4e5d-9c0e-a061bb5bfbb2 admin admin] PATH_INFO: `/v3/projects/2052ef42be464cc98e98aa5da24ee7ae` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:59:04.701246 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf389b17-075d-4e5d-9c0e-a061bb5bfbb2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2052ef42be464cc98e98aa5da24ee7ae)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:59:04.703193 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf389b17-075d-4e5d-9c0e-a061bb5bfbb2 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:59:04.819874 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-bf389b17-075d-4e5d-9c0e-a061bb5bfbb2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2052ef42be464cc98e98aa5da24ee7ae', 'request_id': 'req-bf389b17-075d-4e5d-9c0e-a061bb5bfbb2'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:59:04.825949 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-bf389b17-075d-4e5d-9c0e-a061bb5bfbb2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2052ef42be464cc98e98aa5da24ee7ae', 'request_id': 'req-bf389b17-075d-4e5d-9c0e-a061bb5bfbb2'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:59:04.893492 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-bf389b17-075d-4e5d-9c0e-a061bb5bfbb2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2052ef42be464cc98e98aa5da24ee7ae', 'request_id': 'req-bf389b17-075d-4e5d-9c0e-a061bb5bfbb2'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:59:04.904421 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-bf389b17-075d-4e5d-9c0e-a061bb5bfbb2 admin admin] The token cache is being invalidate because project 2052ef42be464cc98e98aa5da24ee7ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:59:04.904690 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-bf389b17-075d-4e5d-9c0e-a061bb5bfbb2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:59:04.905348 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-bf389b17-075d-4e5d-9c0e-a061bb5bfbb2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:59:04.907572 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1369/2733] 10.222.0.81 () {58 vars in 1232 bytes} [Wed Jul 20 22:59:04 2022] DELETE /identity/v3/projects/2052ef42be464cc98e98aa5da24ee7ae => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:59:04.912221 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0628a7ff-3906-49b4-b7d3-2d0ba2626093 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:59:04.912562 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0628a7ff-3906-49b4-b7d3-2d0ba2626093 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:59:04.912848 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0628a7ff-3906-49b4-b7d3-2d0ba2626093 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:59:04.930619 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-0628a7ff-3906-49b4-b7d3-2d0ba2626093 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:59:04.931851 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-0628a7ff-3906-49b4-b7d3-2d0ba2626093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:59:04.992503 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1365/2734] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:59:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:59:05.000289 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519eb4f2-55d3-4d8c-90f5-17957d641a31 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:59:05.000588 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519eb4f2-55d3-4d8c-90f5-17957d641a31 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:59:05.000924 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519eb4f2-55d3-4d8c-90f5-17957d641a31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:59:05.020342 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-519eb4f2-55d3-4d8c-90f5-17957d641a31 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:59:05.021565 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-519eb4f2-55d3-4d8c-90f5-17957d641a31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:59:05.029511 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1370/2735] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:59:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:59:05.033597 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0628a7ff-3906-49b4-b7d3-2d0ba2626093 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:59:05.043872 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0628a7ff-3906-49b4-b7d3-2d0ba2626093 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:59:05.048429 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9e5381-a302-4289-96bd-c6364822cb86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:59:05.049100 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9e5381-a302-4289-96bd-c6364822cb86 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:59:05.049375 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9e5381-a302-4289-96bd-c6364822cb86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:59:05.049652 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9e5381-a302-4289-96bd-c6364822cb86 admin admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:59:05.050380 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9e5381-a302-4289-96bd-c6364822cb86 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:59:05.050691 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9e5381-a302-4289-96bd-c6364822cb86 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:59:05.052335 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9e5381-a302-4289-96bd-c6364822cb86 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:59:05.060713 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:59:05.063915 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1366/2736] 10.222.0.81 () {58 vars in 1152 bytes} [Wed Jul 20 22:59:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:59:05.068257 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519eb4f2-55d3-4d8c-90f5-17957d641a31 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:59:05.084999 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519eb4f2-55d3-4d8c-90f5-17957d641a31 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:59:05.084999 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec09a81-42bf-4d6c-a70f-5e2c63e1358c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:59:05.084999 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec09a81-42bf-4d6c-a70f-5e2c63e1358c admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:59:05.084999 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec09a81-42bf-4d6c-a70f-5e2c63e1358c admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:59:05.085419 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec09a81-42bf-4d6c-a70f-5e2c63e1358c admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:59:05.085419 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec09a81-42bf-4d6c-a70f-5e2c63e1358c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:59:05.087131 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec09a81-42bf-4d6c-a70f-5e2c63e1358c admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:59:05.187850 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1371/2737] 10.222.0.81 () {60 vars in 1152 bytes} [Wed Jul 20 22:59:05 2022] POST /identity/v3/projects => generated 385 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:59:05.192124 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9e5381-a302-4289-96bd-c6364822cb86 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:59:05.202379 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9e5381-a302-4289-96bd-c6364822cb86 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:59:05.257926 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2afa1af-45b3-48be-bdfb-d8d491106187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:59:05.258617 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2afa1af-45b3-48be-bdfb-d8d491106187 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:59:05.258899 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2afa1af-45b3-48be-bdfb-d8d491106187 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:59:05.259211 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2afa1af-45b3-48be-bdfb-d8d491106187 admin admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:59:05.259904 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2afa1af-45b3-48be-bdfb-d8d491106187 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:59:05.263282 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2afa1af-45b3-48be-bdfb-d8d491106187 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:59:05.376834 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1367/2738] 10.222.0.81 () {60 vars in 1143 bytes} [Wed Jul 20 22:59:05 2022] POST /identity/v3/users => generated 293 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:59:05.378145 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec09a81-42bf-4d6c-a70f-5e2c63e1358c admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:59:05.389063 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec09a81-42bf-4d6c-a70f-5e2c63e1358c admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:59:05.393706 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36768f7-a18f-401a-9549-5bab11e98672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:59:05.394392 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36768f7-a18f-401a-9549-5bab11e98672 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:59:05.394699 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36768f7-a18f-401a-9549-5bab11e98672 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:59:05.395622 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36768f7-a18f-401a-9549-5bab11e98672 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:59:05.396351 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36768f7-a18f-401a-9549-5bab11e98672 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:59:05.397945 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36768f7-a18f-401a-9549-5bab11e98672 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:59:05.408222 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1372/2739] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:59:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 22:59:05.412133 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2afa1af-45b3-48be-bdfb-d8d491106187 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:59:05.422118 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2afa1af-45b3-48be-bdfb-d8d491106187 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:59:05.427123 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fc1910-17b5-4329-b2d0-d5a79c2b3829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:59:05.427791 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fc1910-17b5-4329-b2d0-d5a79c2b3829 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:59:05.428109 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fc1910-17b5-4329-b2d0-d5a79c2b3829 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:59:05.428418 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fc1910-17b5-4329-b2d0-d5a79c2b3829 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:59:05.429148 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fc1910-17b5-4329-b2d0-d5a79c2b3829 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:59:05.430813 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fc1910-17b5-4329-b2d0-d5a79c2b3829 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:59:05.441163 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1368/2740] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:59:05 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:59:05.445562 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36768f7-a18f-401a-9549-5bab11e98672 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:59:05.455422 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36768f7-a18f-401a-9549-5bab11e98672 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:59:05.460822 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99be232e-a721-4909-83ce-e39bde58b5f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:59:05.461746 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99be232e-a721-4909-83ce-e39bde58b5f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:59:05.462036 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99be232e-a721-4909-83ce-e39bde58b5f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:59:05.462320 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99be232e-a721-4909-83ce-e39bde58b5f7 admin admin] PATH_INFO: `/v3/projects/2e81e10539ec4678a9c968a0e352c2da/users/ad82540cdf114013bb7b7127b8476ff8/roles/559942c4de1744169f8b5e0404ecde6b` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:59:05.476777 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99be232e-a721-4909-83ce-e39bde58b5f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e81e10539ec4678a9c968a0e352c2da, user_id=ad82540cdf114013bb7b7127b8476ff8, role_id=559942c4de1744169f8b5e0404ecde6b)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:59:05.479501 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99be232e-a721-4909-83ce-e39bde58b5f7 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:59:05.596823 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1373/2741] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:59:05 2022] PUT /identity/v3/projects/2e81e10539ec4678a9c968a0e352c2da/users/ad82540cdf114013bb7b7127b8476ff8/roles/559942c4de1744169f8b5e0404ecde6b => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:59:05.607131 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b536f701-2ea0-43b2-ae67-7fc16d8f2f38 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:59:05.607453 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b536f701-2ea0-43b2-ae67-7fc16d8f2f38 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:59:05.607739 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b536f701-2ea0-43b2-ae67-7fc16d8f2f38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:59:05.634143 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-b536f701-2ea0-43b2-ae67-7fc16d8f2f38 None None] MFA Rules not processed for user `ad82540cdf114013bb7b7127b8476ff8`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:59:05.635645 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-b536f701-2ea0-43b2-ae67-7fc16d8f2f38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:59:05.700502 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1369/2742] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:59:05 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:59:05.712159 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99be232e-a721-4909-83ce-e39bde58b5f7 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:59:05.713903 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-99be232e-a721-4909-83ce-e39bde58b5f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:59:05.728133 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:59:05.770420 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99be232e-a721-4909-83ce-e39bde58b5f7 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:59:05.774293 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8d6c9c-f8c2-47e9-af56-94786daa75c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecf4db289fba4e4a806198c62d31f7c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:59:05.774951 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8d6c9c-f8c2-47e9-af56-94786daa75c0 service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:59:05.775323 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8d6c9c-f8c2-47e9-af56-94786daa75c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:59:05.775604 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8d6c9c-f8c2-47e9-af56-94786daa75c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:59:05.783996 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8d6c9c-f8c2-47e9-af56-94786daa75c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:59:05.785421 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8d6c9c-f8c2-47e9-af56-94786daa75c0 service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:59:05.794559 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1374/2743] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:59:05 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:59:09.898736 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b536f701-2ea0-43b2-ae67-7fc16d8f2f38 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:59:09.989134 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b536f701-2ea0-43b2-ae67-7fc16d8f2f38 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:59:09.994857 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7664a23e-1b15-4b04-a4c0-98ae877e4797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:59:09.995785 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7664a23e-1b15-4b04-a4c0-98ae877e4797 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:59:09.996070 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7664a23e-1b15-4b04-a4c0-98ae877e4797 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:59:09.996349 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7664a23e-1b15-4b04-a4c0-98ae877e4797 admin admin] PATH_INFO: `/v3/users/ad82540cdf114013bb7b7127b8476ff8` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:59:09.997542 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7664a23e-1b15-4b04-a4c0-98ae877e4797 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad82540cdf114013bb7b7127b8476ff8)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:59:09.999488 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7664a23e-1b15-4b04-a4c0-98ae877e4797 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:59:10.011005 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:59:10.142164 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-7664a23e-1b15-4b04-a4c0-98ae877e4797 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad82540cdf114013bb7b7127b8476ff8'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:59:10.142833 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-7664a23e-1b15-4b04-a4c0-98ae877e4797 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad82540cdf114013bb7b7127b8476ff8'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:59:10.143477 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-7664a23e-1b15-4b04-a4c0-98ae877e4797 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad82540cdf114013bb7b7127b8476ff8'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:59:10.151321 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-7664a23e-1b15-4b04-a4c0-98ae877e4797 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad82540cdf114013bb7b7127b8476ff8'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:59:10.213655 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-7664a23e-1b15-4b04-a4c0-98ae877e4797 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad82540cdf114013bb7b7127b8476ff8'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:59:10.232030 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1370/2744] 10.222.0.81 () {58 vars in 1223 bytes} [Wed Jul 20 22:59:09 2022] DELETE /identity/v3/users/ad82540cdf114013bb7b7127b8476ff8 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:59:10.236534 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc7be9a-caad-4a29-900f-8b17e7919cac None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:59:10.236999 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc7be9a-caad-4a29-900f-8b17e7919cac None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:59:10.237105 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc7be9a-caad-4a29-900f-8b17e7919cac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:59:10.254220 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-fdc7be9a-caad-4a29-900f-8b17e7919cac None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:59:10.255483 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-fdc7be9a-caad-4a29-900f-8b17e7919cac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:59:10.316709 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1375/2745] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:59:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:59:10.326413 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7664a23e-1b15-4b04-a4c0-98ae877e4797 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:59:10.327842 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-7664a23e-1b15-4b04-a4c0-98ae877e4797 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:59:10.384134 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7664a23e-1b15-4b04-a4c0-98ae877e4797 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:59:10.388036 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abc3bff-1ad9-4acc-a6e1-735e59772cc0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:59:10.388677 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abc3bff-1ad9-4acc-a6e1-735e59772cc0 service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:59:10.388964 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abc3bff-1ad9-4acc-a6e1-735e59772cc0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:59:10.389245 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abc3bff-1ad9-4acc-a6e1-735e59772cc0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:59:10.398745 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abc3bff-1ad9-4acc-a6e1-735e59772cc0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:59:10.402949 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abc3bff-1ad9-4acc-a6e1-735e59772cc0 service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:59:10.409729 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1371/2746] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:59:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:59:10.799895 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc7be9a-caad-4a29-900f-8b17e7919cac None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:59:10.800387 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-fdc7be9a-caad-4a29-900f-8b17e7919cac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:59:10.812398 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc7be9a-caad-4a29-900f-8b17e7919cac None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:59:10.818015 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b85b050-9dc2-4683-8e91-2e3a938a50fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:59:10.818981 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b85b050-9dc2-4683-8e91-2e3a938a50fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:59:10.819304 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b85b050-9dc2-4683-8e91-2e3a938a50fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:59:10.819584 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b85b050-9dc2-4683-8e91-2e3a938a50fc admin admin] PATH_INFO: `/v3/projects/2e81e10539ec4678a9c968a0e352c2da` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:59:10.820778 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b85b050-9dc2-4683-8e91-2e3a938a50fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e81e10539ec4678a9c968a0e352c2da)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:59:10.822685 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b85b050-9dc2-4683-8e91-2e3a938a50fc admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:59:10.974491 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-0b85b050-9dc2-4683-8e91-2e3a938a50fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e81e10539ec4678a9c968a0e352c2da', 'request_id': 'req-0b85b050-9dc2-4683-8e91-2e3a938a50fc'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:59:10.980568 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-0b85b050-9dc2-4683-8e91-2e3a938a50fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e81e10539ec4678a9c968a0e352c2da', 'request_id': 'req-0b85b050-9dc2-4683-8e91-2e3a938a50fc'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:59:11.070643 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-0b85b050-9dc2-4683-8e91-2e3a938a50fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e81e10539ec4678a9c968a0e352c2da', 'request_id': 'req-0b85b050-9dc2-4683-8e91-2e3a938a50fc'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:59:11.081407 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-0b85b050-9dc2-4683-8e91-2e3a938a50fc admin admin] The token cache is being invalidate because project 2e81e10539ec4678a9c968a0e352c2da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:59:11.081698 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-0b85b050-9dc2-4683-8e91-2e3a938a50fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:59:11.082384 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-0b85b050-9dc2-4683-8e91-2e3a938a50fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:59:11.083961 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1376/2747] 10.222.0.81 () {58 vars in 1232 bytes} [Wed Jul 20 22:59:10 2022] DELETE /identity/v3/projects/2e81e10539ec4678a9c968a0e352c2da => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:59:11.089004 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33150a7-8867-4277-9244-ce215389ca98 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:59:11.089506 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33150a7-8867-4277-9244-ce215389ca98 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:59:11.089709 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33150a7-8867-4277-9244-ce215389ca98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:59:11.107409 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-b33150a7-8867-4277-9244-ce215389ca98 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:59:11.108618 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-b33150a7-8867-4277-9244-ce215389ca98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:59:11.168011 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1372/2748] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:59:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:59:11.176908 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf23388-cc39-4199-86fd-40eb61cfd527 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:59:11.177200 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf23388-cc39-4199-86fd-40eb61cfd527 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:59:11.177481 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf23388-cc39-4199-86fd-40eb61cfd527 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:59:11.199642 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-adf23388-cc39-4199-86fd-40eb61cfd527 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:59:11.199642 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-adf23388-cc39-4199-86fd-40eb61cfd527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:59:11.206362 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1377/2749] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:59:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:59:11.211141 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33150a7-8867-4277-9244-ce215389ca98 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:59:11.220875 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33150a7-8867-4277-9244-ce215389ca98 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:59:11.226156 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4ee1be-d8a5-413e-bcd7-34ea6b0c3b1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:59:11.226754 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4ee1be-d8a5-413e-bcd7-34ea6b0c3b1f admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:59:11.227064 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4ee1be-d8a5-413e-bcd7-34ea6b0c3b1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:59:11.227346 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4ee1be-d8a5-413e-bcd7-34ea6b0c3b1f admin admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:59:11.228062 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4ee1be-d8a5-413e-bcd7-34ea6b0c3b1f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 22:59:11.228370 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4ee1be-d8a5-413e-bcd7-34ea6b0c3b1f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:59:11.229955 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4ee1be-d8a5-413e-bcd7-34ea6b0c3b1f admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:59:11.241321 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1373/2750] 10.222.0.81 () {58 vars in 1152 bytes} [Wed Jul 20 22:59:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 22:59:11.245552 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf23388-cc39-4199-86fd-40eb61cfd527 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:59:11.254601 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf23388-cc39-4199-86fd-40eb61cfd527 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:59:11.259266 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af5c655-ac4d-44ca-b33d-4769a82900a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:59:11.259945 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af5c655-ac4d-44ca-b33d-4769a82900a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:59:11.260488 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af5c655-ac4d-44ca-b33d-4769a82900a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:59:11.260760 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af5c655-ac4d-44ca-b33d-4769a82900a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:59:11.261435 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af5c655-ac4d-44ca-b33d-4769a82900a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:59:11.263326 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af5c655-ac4d-44ca-b33d-4769a82900a8 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:59:11.353401 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1378/2751] 10.222.0.81 () {60 vars in 1152 bytes} [Wed Jul 20 22:59:11 2022] POST /identity/v3/projects => generated 375 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:59:11.357716 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4ee1be-d8a5-413e-bcd7-34ea6b0c3b1f admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:59:11.367955 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4ee1be-d8a5-413e-bcd7-34ea6b0c3b1f admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:59:11.380845 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:59:11.424189 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad3842f-6668-4a51-b1d3-a096f7f7adc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:59:11.424890 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad3842f-6668-4a51-b1d3-a096f7f7adc6 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:59:11.425180 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad3842f-6668-4a51-b1d3-a096f7f7adc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:59:11.425463 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad3842f-6668-4a51-b1d3-a096f7f7adc6 admin admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:59:11.426164 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad3842f-6668-4a51-b1d3-a096f7f7adc6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:59:11.428136 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad3842f-6668-4a51-b1d3-a096f7f7adc6 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:59:11.534046 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1374/2752] 10.222.0.81 () {60 vars in 1143 bytes} [Wed Jul 20 22:59:11 2022] POST /identity/v3/users => generated 288 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 22:59:11.538962 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af5c655-ac4d-44ca-b33d-4769a82900a8 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:59:11.549364 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af5c655-ac4d-44ca-b33d-4769a82900a8 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:59:11.554518 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3078b7-9865-46ed-9e36-554d46e6ba30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:59:11.555290 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3078b7-9865-46ed-9e36-554d46e6ba30 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:59:11.555588 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3078b7-9865-46ed-9e36-554d46e6ba30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:59:11.555864 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3078b7-9865-46ed-9e36-554d46e6ba30 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:59:11.556588 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3078b7-9865-46ed-9e36-554d46e6ba30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:59:11.558156 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3078b7-9865-46ed-9e36-554d46e6ba30 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:59:11.563257 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:59:11.568650 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1379/2753] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:59:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 22:59:11.575426 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad3842f-6668-4a51-b1d3-a096f7f7adc6 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:59:11.585704 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad3842f-6668-4a51-b1d3-a096f7f7adc6 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:59:11.590531 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3f85f1-cb34-4512-a96a-663d301f7541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:59:11.591298 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3f85f1-cb34-4512-a96a-663d301f7541 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:59:11.591589 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3f85f1-cb34-4512-a96a-663d301f7541 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:59:11.591870 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3f85f1-cb34-4512-a96a-663d301f7541 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:59:11.592658 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3f85f1-cb34-4512-a96a-663d301f7541 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:59:11.594292 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3f85f1-cb34-4512-a96a-663d301f7541 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:59:11.604711 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1375/2754] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 22:59:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 22:59:11.608732 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3078b7-9865-46ed-9e36-554d46e6ba30 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:59:11.619375 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3078b7-9865-46ed-9e36-554d46e6ba30 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:59:11.624552 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479c9e72-97a7-483f-929b-c6df209a04c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:59:11.625454 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479c9e72-97a7-483f-929b-c6df209a04c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:59:11.625733 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479c9e72-97a7-483f-929b-c6df209a04c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:59:11.626005 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479c9e72-97a7-483f-929b-c6df209a04c2 admin admin] PATH_INFO: `/v3/projects/e78d1ca7276f47fb89822044d62ac720/users/9242b67e7c0b4115ae4ffc8c37a12ea9/roles/559942c4de1744169f8b5e0404ecde6b` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:59:11.641007 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479c9e72-97a7-483f-929b-c6df209a04c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e78d1ca7276f47fb89822044d62ac720, user_id=9242b67e7c0b4115ae4ffc8c37a12ea9, role_id=559942c4de1744169f8b5e0404ecde6b)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:59:11.643736 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479c9e72-97a7-483f-929b-c6df209a04c2 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:59:11.710729 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1380/2755] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 22:59:11 2022] PUT /identity/v3/projects/e78d1ca7276f47fb89822044d62ac720/users/9242b67e7c0b4115ae4ffc8c37a12ea9/roles/559942c4de1744169f8b5e0404ecde6b => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:59:11.721211 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3868f6ac-1e64-4f71-af04-00000fd84de4 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:59:11.721533 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3868f6ac-1e64-4f71-af04-00000fd84de4 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:59:11.721819 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3868f6ac-1e64-4f71-af04-00000fd84de4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:59:11.749095 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-3868f6ac-1e64-4f71-af04-00000fd84de4 None None] MFA Rules not processed for user `9242b67e7c0b4115ae4ffc8c37a12ea9`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:59:11.750317 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-3868f6ac-1e64-4f71-af04-00000fd84de4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:59:11.813298 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1376/2756] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:59:11 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:59:11.824081 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479c9e72-97a7-483f-929b-c6df209a04c2 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:59:11.825775 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-479c9e72-97a7-483f-929b-c6df209a04c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:59:11.882037 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479c9e72-97a7-483f-929b-c6df209a04c2 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:59:11.885970 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a589cdf-8e6a-4c96-b856-3fd0f12334a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecf4db289fba4e4a806198c62d31f7c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:59:11.886618 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a589cdf-8e6a-4c96-b856-3fd0f12334a8 service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:59:11.886902 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a589cdf-8e6a-4c96-b856-3fd0f12334a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:59:11.887207 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a589cdf-8e6a-4c96-b856-3fd0f12334a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:59:11.895419 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a589cdf-8e6a-4c96-b856-3fd0f12334a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:59:11.896845 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a589cdf-8e6a-4c96-b856-3fd0f12334a8 service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:59:11.905619 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1381/2757] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:59:11 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:59:14.900623 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d83bc94-6204-4597-b1ac-7009393b4edd None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:59:14.901100 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d83bc94-6204-4597-b1ac-7009393b4edd None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:59:14.901212 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d83bc94-6204-4597-b1ac-7009393b4edd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:59:14.919655 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-6d83bc94-6204-4597-b1ac-7009393b4edd None None] MFA Rules not processed for user `9242b67e7c0b4115ae4ffc8c37a12ea9`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:59:14.920850 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-6d83bc94-6204-4597-b1ac-7009393b4edd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:59:14.928200 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1377/2758] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:59:14 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:59:21.901602 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=36,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:59:24.913334 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=61,keystone:INSERT=3 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:59:27.909595 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a589cdf-8e6a-4c96-b856-3fd0f12334a8 service cinder] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:59:27.911134 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-5a589cdf-8e6a-4c96-b856-3fd0f12334a8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:59:27.983539 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a589cdf-8e6a-4c96-b856-3fd0f12334a8 service cinder] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:59:27.989106 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c227b119-3995-40a8-89a3-239a1e68c3e1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7829324db2574d38b18c749d0559177f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:59:27.989756 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c227b119-3995-40a8-89a3-239a1e68c3e1 service nova] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:59:27.990040 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c227b119-3995-40a8-89a3-239a1e68c3e1 service nova] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:59:27.990317 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c227b119-3995-40a8-89a3-239a1e68c3e1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:59:27.992160 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-c227b119-3995-40a8-89a3-239a1e68c3e1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:59:28.052008 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c227b119-3995-40a8-89a3-239a1e68c3e1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:59:28.053648 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c227b119-3995-40a8-89a3-239a1e68c3e1 service nova] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:59:28.062731 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1382/2759] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:59:27 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:59:28.351929 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d83bc94-6204-4597-b1ac-7009393b4edd None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:59:28.353387 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-6d83bc94-6204-4597-b1ac-7009393b4edd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:59:28.429623 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d83bc94-6204-4597-b1ac-7009393b4edd None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:59:28.435257 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3424b411-6d75-4cf1-987b-cdb18841ea3c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5500cebdd9d4f8e9aecb45d6c955878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:59:28.435921 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3424b411-6d75-4cf1-987b-cdb18841ea3c service placement] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:59:28.436203 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3424b411-6d75-4cf1-987b-cdb18841ea3c service placement] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:59:28.436487 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3424b411-6d75-4cf1-987b-cdb18841ea3c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:59:28.438667 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-3424b411-6d75-4cf1-987b-cdb18841ea3c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:59:28.450754 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3424b411-6d75-4cf1-987b-cdb18841ea3c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:59:28.454272 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3424b411-6d75-4cf1-987b-cdb18841ea3c service placement] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:59:28.462826 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1378/2760] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:59:28 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:59:33.629415 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b845fd22-241a-4fe4-86b4-ede9a45b7152 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:59:33.629929 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b845fd22-241a-4fe4-86b4-ede9a45b7152 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:59:33.630024 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b845fd22-241a-4fe4-86b4-ede9a45b7152 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:59:33.647758 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-b845fd22-241a-4fe4-86b4-ede9a45b7152 None None] MFA Rules not processed for user `90915e1264324f16bc7bae7e46e0c97b`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:59:33.649008 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-b845fd22-241a-4fe4-86b4-ede9a45b7152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:59:33.704752 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1383/2761] 10.222.0.81 () {58 vars in 1009 bytes} [Wed Jul 20 22:59:33 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:59:33.717010 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3424b411-6d75-4cf1-987b-cdb18841ea3c service placement] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:59:33.718471 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-3424b411-6d75-4cf1-987b-cdb18841ea3c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:59:33.786597 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3424b411-6d75-4cf1-987b-cdb18841ea3c service placement] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:59:33.791524 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2103943-0df0-40d1-b87c-d3ffe315716a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70e97436a08543a180225561ef8bc817', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:59:33.792170 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2103943-0df0-40d1-b87c-d3ffe315716a service swift] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:59:33.792465 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2103943-0df0-40d1-b87c-d3ffe315716a service swift] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:59:33.792747 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2103943-0df0-40d1-b87c-d3ffe315716a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:59:33.800863 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2103943-0df0-40d1-b87c-d3ffe315716a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:59:33.802445 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2103943-0df0-40d1-b87c-d3ffe315716a service swift] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:59:33.810492 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1379/2762] 10.222.0.81 () {60 vars in 1384 bytes} [Wed Jul 20 22:59:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 22:59:34.775193 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd3c29b-736c-4a38-ac0c-db18b4b20fdc None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:59:34.775685 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd3c29b-736c-4a38-ac0c-db18b4b20fdc None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:59:34.775798 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd3c29b-736c-4a38-ac0c-db18b4b20fdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:59:34.793329 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-bbd3c29b-736c-4a38-ac0c-db18b4b20fdc None None] MFA Rules not processed for user `90915e1264324f16bc7bae7e46e0c97b`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:59:34.794530 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-bbd3c29b-736c-4a38-ac0c-db18b4b20fdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:59:34.801670 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1384/2763] 10.222.0.81 () {58 vars in 967 bytes} [Wed Jul 20 22:59:34 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:59:34.813732 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2103943-0df0-40d1-b87c-d3ffe315716a service swift] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:59:34.824884 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2103943-0df0-40d1-b87c-d3ffe315716a service swift] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:59:34.830110 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52cf28a-13a4-4b82-9767-1cb32af1e608 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70e97436a08543a180225561ef8bc817', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:59:34.830792 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52cf28a-13a4-4b82-9767-1cb32af1e608 service swift] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:59:34.831134 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52cf28a-13a4-4b82-9767-1cb32af1e608 service swift] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:59:34.831416 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52cf28a-13a4-4b82-9767-1cb32af1e608 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:59:34.840327 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52cf28a-13a4-4b82-9767-1cb32af1e608 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:59:34.841981 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52cf28a-13a4-4b82-9767-1cb32af1e608 service swift] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:59:34.850244 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1380/2764] 10.222.0.81 () {60 vars in 1384 bytes} [Wed Jul 20 22:59:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 22:59:35.542951 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768b19e3-36ff-4b12-9cfb-29a5aaaa6dd2 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:59:35.549329 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768b19e3-36ff-4b12-9cfb-29a5aaaa6dd2 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:59:35.549329 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768b19e3-36ff-4b12-9cfb-29a5aaaa6dd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:59:35.561519 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-768b19e3-36ff-4b12-9cfb-29a5aaaa6dd2 None None] MFA Rules not processed for user `90915e1264324f16bc7bae7e46e0c97b`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:59:35.562712 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-768b19e3-36ff-4b12-9cfb-29a5aaaa6dd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:59:35.569977 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1385/2765] 10.222.0.81 () {58 vars in 1009 bytes} [Wed Jul 20 22:59:35 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:59:35.581623 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52cf28a-13a4-4b82-9767-1cb32af1e608 service swift] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:59:35.592135 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52cf28a-13a4-4b82-9767-1cb32af1e608 service swift] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:59:35.597201 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cb0c01-ff24-4998-b651-ce38412d3122 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70e97436a08543a180225561ef8bc817', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:59:35.598355 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cb0c01-ff24-4998-b651-ce38412d3122 service swift] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:59:35.598638 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cb0c01-ff24-4998-b651-ce38412d3122 service swift] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:59:35.598915 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cb0c01-ff24-4998-b651-ce38412d3122 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:59:35.607102 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cb0c01-ff24-4998-b651-ce38412d3122 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:59:35.608679 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cb0c01-ff24-4998-b651-ce38412d3122 service swift] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:59:35.617351 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1381/2766] 10.222.0.81 () {60 vars in 1384 bytes} [Wed Jul 20 22:59:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 22:59:36.245611 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b3f6ec-3f3d-465a-9393-8626396adb20 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:59:36.246085 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b3f6ec-3f3d-465a-9393-8626396adb20 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:59:36.246182 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b3f6ec-3f3d-465a-9393-8626396adb20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:59:36.264468 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-75b3f6ec-3f3d-465a-9393-8626396adb20 None None] MFA Rules not processed for user `90915e1264324f16bc7bae7e46e0c97b`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:59:36.265665 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-75b3f6ec-3f3d-465a-9393-8626396adb20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:59:36.273543 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1386/2767] 10.222.0.81 () {58 vars in 967 bytes} [Wed Jul 20 22:59:36 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:59:36.285327 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cb0c01-ff24-4998-b651-ce38412d3122 service swift] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:59:36.295988 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cb0c01-ff24-4998-b651-ce38412d3122 service swift] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:59:36.300900 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09498a7c-8261-4b2d-bff4-079dc3d6f4e8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70e97436a08543a180225561ef8bc817', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:59:36.301522 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09498a7c-8261-4b2d-bff4-079dc3d6f4e8 service swift] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:59:36.301866 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09498a7c-8261-4b2d-bff4-079dc3d6f4e8 service swift] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:59:36.302148 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09498a7c-8261-4b2d-bff4-079dc3d6f4e8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:59:36.310260 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09498a7c-8261-4b2d-bff4-079dc3d6f4e8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:59:36.311866 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09498a7c-8261-4b2d-bff4-079dc3d6f4e8 service swift] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:59:36.320252 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1382/2768] 10.222.0.81 () {60 vars in 1384 bytes} [Wed Jul 20 22:59:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 20 22:59:38.954603 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b3f6ec-3f3d-465a-9393-8626396adb20 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:59:38.980410 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=100 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:59:39.017676 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b3f6ec-3f3d-465a-9393-8626396adb20 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:59:39.022953 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dc518c-b50f-4f8f-a094-5c5c461ac617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:59:39.023888 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dc518c-b50f-4f8f-a094-5c5c461ac617 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:59:39.024673 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dc518c-b50f-4f8f-a094-5c5c461ac617 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:59:39.024973 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dc518c-b50f-4f8f-a094-5c5c461ac617 admin admin] PATH_INFO: `/v3/users/9242b67e7c0b4115ae4ffc8c37a12ea9` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:59:39.026351 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dc518c-b50f-4f8f-a094-5c5c461ac617 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9242b67e7c0b4115ae4ffc8c37a12ea9)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:59:39.028344 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dc518c-b50f-4f8f-a094-5c5c461ac617 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:59:39.234442 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-29dc518c-b50f-4f8f-a094-5c5c461ac617 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9242b67e7c0b4115ae4ffc8c37a12ea9'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:59:39.235177 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-29dc518c-b50f-4f8f-a094-5c5c461ac617 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9242b67e7c0b4115ae4ffc8c37a12ea9'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:59:39.239453 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-29dc518c-b50f-4f8f-a094-5c5c461ac617 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9242b67e7c0b4115ae4ffc8c37a12ea9'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:59:39.243266 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-29dc518c-b50f-4f8f-a094-5c5c461ac617 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9242b67e7c0b4115ae4ffc8c37a12ea9'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:59:39.321793 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-29dc518c-b50f-4f8f-a094-5c5c461ac617 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9242b67e7c0b4115ae4ffc8c37a12ea9'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:59:39.339724 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1387/2769] 10.222.0.81 () {58 vars in 1223 bytes} [Wed Jul 20 22:59:38 2022] DELETE /identity/v3/users/9242b67e7c0b4115ae4ffc8c37a12ea9 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 22:59:39.344319 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321a0fbf-b70e-4cc3-96d3-5f1d31cf1bd1 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:59:39.344853 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321a0fbf-b70e-4cc3-96d3-5f1d31cf1bd1 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:59:39.344853 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321a0fbf-b70e-4cc3-96d3-5f1d31cf1bd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:59:39.362428 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-321a0fbf-b70e-4cc3-96d3-5f1d31cf1bd1 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:59:39.364285 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-321a0fbf-b70e-4cc3-96d3-5f1d31cf1bd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:59:39.396758 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=100 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:59:39.422766 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1383/2770] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 22:59:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:59:39.433015 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dc518c-b50f-4f8f-a094-5c5c461ac617 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:59:39.434449 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-29dc518c-b50f-4f8f-a094-5c5c461ac617 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:59:39.492456 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dc518c-b50f-4f8f-a094-5c5c461ac617 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:59:39.496071 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d20ca7a-c248-4dda-9a07-7b78d825f213 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:59:39.496721 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d20ca7a-c248-4dda-9a07-7b78d825f213 service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:59:39.497011 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d20ca7a-c248-4dda-9a07-7b78d825f213 service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:59:39.497295 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d20ca7a-c248-4dda-9a07-7b78d825f213 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:59:39.506546 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d20ca7a-c248-4dda-9a07-7b78d825f213 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:59:39.507998 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d20ca7a-c248-4dda-9a07-7b78d825f213 service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:59:39.517567 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1388/2771] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 22:59:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 22:59:39.792143 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321a0fbf-b70e-4cc3-96d3-5f1d31cf1bd1 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:59:39.796648 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-321a0fbf-b70e-4cc3-96d3-5f1d31cf1bd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:59:39.806059 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321a0fbf-b70e-4cc3-96d3-5f1d31cf1bd1 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:59:39.810822 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83ecdb2-0417-401b-a1b5-cc8160f1a1d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:59:39.815043 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83ecdb2-0417-401b-a1b5-cc8160f1a1d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:59:39.815043 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83ecdb2-0417-401b-a1b5-cc8160f1a1d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:59:39.815043 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83ecdb2-0417-401b-a1b5-cc8160f1a1d5 admin admin] PATH_INFO: `/v3/projects/e78d1ca7276f47fb89822044d62ac720` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:59:39.815043 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83ecdb2-0417-401b-a1b5-cc8160f1a1d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e78d1ca7276f47fb89822044d62ac720)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:59:39.816027 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83ecdb2-0417-401b-a1b5-cc8160f1a1d5 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:59:39.944882 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-c83ecdb2-0417-401b-a1b5-cc8160f1a1d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e78d1ca7276f47fb89822044d62ac720', 'request_id': 'req-c83ecdb2-0417-401b-a1b5-cc8160f1a1d5'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:59:40.036761 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-c83ecdb2-0417-401b-a1b5-cc8160f1a1d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e78d1ca7276f47fb89822044d62ac720', 'request_id': 'req-c83ecdb2-0417-401b-a1b5-cc8160f1a1d5'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:59:40.043162 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-c83ecdb2-0417-401b-a1b5-cc8160f1a1d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e78d1ca7276f47fb89822044d62ac720', 'request_id': 'req-c83ecdb2-0417-401b-a1b5-cc8160f1a1d5'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:59:40.055121 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-c83ecdb2-0417-401b-a1b5-cc8160f1a1d5 admin admin] The token cache is being invalidate because project e78d1ca7276f47fb89822044d62ac720 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 22:59:40.055395 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-c83ecdb2-0417-401b-a1b5-cc8160f1a1d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:59:40.056023 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-c83ecdb2-0417-401b-a1b5-cc8160f1a1d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 22:59:40.057587 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1384/2772] 10.222.0.81 () {58 vars in 1232 bytes} [Wed Jul 20 22:59:39 2022] DELETE /identity/v3/projects/e78d1ca7276f47fb89822044d62ac720 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 22:59:49.513458 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=69,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:59:50.051962 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=36,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 22:59:54.179776 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79df3b1e-9b93-4844-a407-7429e2902dce None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:59:54.180276 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79df3b1e-9b93-4844-a407-7429e2902dce None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:59:54.180469 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79df3b1e-9b93-4844-a407-7429e2902dce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:59:54.198838 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-79df3b1e-9b93-4844-a407-7429e2902dce None None] MFA Rules not processed for user `90915e1264324f16bc7bae7e46e0c97b`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 22:59:54.200111 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-79df3b1e-9b93-4844-a407-7429e2902dce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:59:54.255283 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1389/2773] 10.222.0.81 () {58 vars in 1009 bytes} [Wed Jul 20 22:59:54 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 22:59:54.276637 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83ecdb2-0417-401b-a1b5-cc8160f1a1d5 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 22:59:54.278089 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-c83ecdb2-0417-401b-a1b5-cc8160f1a1d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 22:59:54.348142 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83ecdb2-0417-401b-a1b5-cc8160f1a1d5 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 22:59:54.353240 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a81de26-f407-451a-8120-ad2d14cd9c63 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70e97436a08543a180225561ef8bc817', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 22:59:54.353901 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a81de26-f407-451a-8120-ad2d14cd9c63 service swift] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 22:59:54.354196 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a81de26-f407-451a-8120-ad2d14cd9c63 service swift] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 22:59:54.354482 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a81de26-f407-451a-8120-ad2d14cd9c63 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 22:59:54.363568 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a81de26-f407-451a-8120-ad2d14cd9c63 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 22:59:54.365169 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a81de26-f407-451a-8120-ad2d14cd9c63 service swift] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 22:59:54.373490 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1385/2774] 10.222.0.81 () {60 vars in 1384 bytes} [Wed Jul 20 22:59:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 23:00:04.247647 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=25 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 23:00:04.370002 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=32 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 23:00:13.243988 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b241a026-65de-4ef3-bcc1-eab0d3cd3f61 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:00:13.244506 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b241a026-65de-4ef3-bcc1-eab0d3cd3f61 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:00:13.244614 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b241a026-65de-4ef3-bcc1-eab0d3cd3f61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:00:13.263167 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-b241a026-65de-4ef3-bcc1-eab0d3cd3f61 None None] MFA Rules not processed for user `90915e1264324f16bc7bae7e46e0c97b`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 23:00:13.264369 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-b241a026-65de-4ef3-bcc1-eab0d3cd3f61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 23:00:13.271600 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1390/2775] 10.222.0.81 () {58 vars in 1009 bytes} [Wed Jul 20 23:00:13 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 23:00:13.283194 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a81de26-f407-451a-8120-ad2d14cd9c63 service swift] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:00:13.293718 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a81de26-f407-451a-8120-ad2d14cd9c63 service swift] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:00:13.298697 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b80a813-df17-4831-b6ab-e80a77902a8f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70e97436a08543a180225561ef8bc817', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:00:13.299376 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b80a813-df17-4831-b6ab-e80a77902a8f service swift] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:00:13.299665 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b80a813-df17-4831-b6ab-e80a77902a8f service swift] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:00:13.300510 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b80a813-df17-4831-b6ab-e80a77902a8f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:00:13.308602 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b80a813-df17-4831-b6ab-e80a77902a8f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:00:13.310175 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b80a813-df17-4831-b6ab-e80a77902a8f service swift] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:00:13.318859 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1386/2776] 10.222.0.81 () {60 vars in 1384 bytes} [Wed Jul 20 23:00:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 23:00:23.257575 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 23:00:23.314992 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=6 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 23:00:26.096497 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b241a026-65de-4ef3-bcc1-eab0d3cd3f61 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:00:26.098008 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-b241a026-65de-4ef3-bcc1-eab0d3cd3f61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 23:00:26.154151 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b241a026-65de-4ef3-bcc1-eab0d3cd3f61 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:00:26.157744 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f685e35-80c2-4e5b-b449-982541db00a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:00:26.158419 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f685e35-80c2-4e5b-b449-982541db00a8 service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:00:26.158712 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f685e35-80c2-4e5b-b449-982541db00a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:00:26.159021 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f685e35-80c2-4e5b-b449-982541db00a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:00:26.162124 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-5f685e35-80c2-4e5b-b449-982541db00a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 23:00:26.232378 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f685e35-80c2-4e5b-b449-982541db00a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:00:26.233750 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f685e35-80c2-4e5b-b449-982541db00a8 service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:00:26.244691 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1391/2777] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 23:00:26 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 23:00:27.418183 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b80a813-df17-4831-b6ab-e80a77902a8f service swift] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:00:27.419623 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-4b80a813-df17-4831-b6ab-e80a77902a8f service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 23:00:27.432192 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b80a813-df17-4831-b6ab-e80a77902a8f service swift] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:00:27.435775 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb15040a-d69a-4764-addc-8095160ff68b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:00:27.436496 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb15040a-d69a-4764-addc-8095160ff68b service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:00:27.436788 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb15040a-d69a-4764-addc-8095160ff68b service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:00:27.437073 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb15040a-d69a-4764-addc-8095160ff68b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:00:27.438783 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-eb15040a-d69a-4764-addc-8095160ff68b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 23:00:27.452547 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb15040a-d69a-4764-addc-8095160ff68b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:00:27.453920 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb15040a-d69a-4764-addc-8095160ff68b service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:00:27.465538 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1387/2778] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 23:00:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 23:00:32.291672 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5d1e77-e058-44cb-bfea-af717392d6cf None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:00:32.292160 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5d1e77-e058-44cb-bfea-af717392d6cf None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:00:32.292266 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5d1e77-e058-44cb-bfea-af717392d6cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:00:32.311815 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-aa5d1e77-e058-44cb-bfea-af717392d6cf None None] MFA Rules not processed for user `90915e1264324f16bc7bae7e46e0c97b`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 23:00:32.313396 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-aa5d1e77-e058-44cb-bfea-af717392d6cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 23:00:32.320633 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1392/2779] 10.222.0.81 () {58 vars in 967 bytes} [Wed Jul 20 23:00:32 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 23:00:32.333227 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb15040a-d69a-4764-addc-8095160ff68b service neutron] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:00:32.344372 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb15040a-d69a-4764-addc-8095160ff68b service neutron] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:00:32.350277 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbb1bf6-e70c-4b41-8d37-4a12facc766b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70e97436a08543a180225561ef8bc817', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:00:32.350862 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbb1bf6-e70c-4b41-8d37-4a12facc766b service swift] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:00:32.351189 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbb1bf6-e70c-4b41-8d37-4a12facc766b service swift] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:00:32.351474 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbb1bf6-e70c-4b41-8d37-4a12facc766b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:00:32.361539 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbb1bf6-e70c-4b41-8d37-4a12facc766b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:00:32.363148 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbb1bf6-e70c-4b41-8d37-4a12facc766b service swift] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:00:32.371379 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1388/2780] 10.222.0.81 () {60 vars in 1384 bytes} [Wed Jul 20 23:00:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 23:00:42.305437 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=57 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 23:00:42.368031 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=12 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 23:01:38.680604 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5d1e77-e058-44cb-bfea-af717392d6cf None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:01:38.690512 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5d1e77-e058-44cb-bfea-af717392d6cf None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:01:38.694050 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e41bd8-47ba-4fee-b634-b5daa881a32d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:01:38.694693 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e41bd8-47ba-4fee-b634-b5daa881a32d service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:01:38.694978 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e41bd8-47ba-4fee-b634-b5daa881a32d service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:01:38.695357 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e41bd8-47ba-4fee-b634-b5daa881a32d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:01:38.697095 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-85e41bd8-47ba-4fee-b634-b5daa881a32d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 23:01:38.759676 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e41bd8-47ba-4fee-b634-b5daa881a32d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:01:38.761106 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e41bd8-47ba-4fee-b634-b5daa881a32d service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:01:38.770634 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1393/2781] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 23:01:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 23:01:42.152659 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbb1bf6-e70c-4b41-8d37-4a12facc766b service swift] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:01:42.154174 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-bcbb1bf6-e70c-4b41-8d37-4a12facc766b service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 23:01:42.231058 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbb1bf6-e70c-4b41-8d37-4a12facc766b service swift] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:01:42.236193 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1942353-eb9a-41c3-9c56-ac1ffefe84db service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5500cebdd9d4f8e9aecb45d6c955878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:01:42.240197 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1942353-eb9a-41c3-9c56-ac1ffefe84db service placement] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:01:42.240197 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1942353-eb9a-41c3-9c56-ac1ffefe84db service placement] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:01:42.240197 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1942353-eb9a-41c3-9c56-ac1ffefe84db service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:01:42.240197 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-d1942353-eb9a-41c3-9c56-ac1ffefe84db service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 23:01:42.252467 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1942353-eb9a-41c3-9c56-ac1ffefe84db service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:01:42.254164 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1942353-eb9a-41c3-9c56-ac1ffefe84db service placement] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:01:42.266283 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1389/2782] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 23:01:42 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 23:01:42.304006 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e41bd8-47ba-4fee-b634-b5daa881a32d service neutron] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:01:42.306191 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-85e41bd8-47ba-4fee-b634-b5daa881a32d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 23:01:42.318912 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e41bd8-47ba-4fee-b634-b5daa881a32d service neutron] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:01:42.324143 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda8ed07-c43a-4520-876c-9c972be6c561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:01:42.325163 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda8ed07-c43a-4520-876c-9c972be6c561 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:01:42.325516 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda8ed07-c43a-4520-876c-9c972be6c561 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:01:42.325866 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda8ed07-c43a-4520-876c-9c972be6c561 admin admin] PATH_INFO: `/v3/users/bc19753d420840ffa5cde19d4c1fac8f` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:01:42.327190 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda8ed07-c43a-4520-876c-9c972be6c561 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc19753d420840ffa5cde19d4c1fac8f)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:01:42.329185 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda8ed07-c43a-4520-876c-9c972be6c561 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:01:42.550170 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-eda8ed07-c43a-4520-876c-9c972be6c561 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc19753d420840ffa5cde19d4c1fac8f'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 23:01:42.550771 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-eda8ed07-c43a-4520-876c-9c972be6c561 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc19753d420840ffa5cde19d4c1fac8f'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 23:01:42.551408 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-eda8ed07-c43a-4520-876c-9c972be6c561 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc19753d420840ffa5cde19d4c1fac8f'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 23:01:42.558250 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-eda8ed07-c43a-4520-876c-9c972be6c561 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc19753d420840ffa5cde19d4c1fac8f'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 23:01:42.615872 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-eda8ed07-c43a-4520-876c-9c972be6c561 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc19753d420840ffa5cde19d4c1fac8f'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 23:01:42.634127 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1394/2783] 10.222.0.81 () {58 vars in 1223 bytes} [Wed Jul 20 23:01:42 2022] DELETE /identity/v3/users/bc19753d420840ffa5cde19d4c1fac8f => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 23:01:43.157846 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1942353-eb9a-41c3-9c56-ac1ffefe84db service placement] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:01:43.174094 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1942353-eb9a-41c3-9c56-ac1ffefe84db service placement] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:01:43.234235 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d67f2a-b63b-4e22-be13-6652ca9b1cd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:01:43.235148 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d67f2a-b63b-4e22-be13-6652ca9b1cd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:01:43.235438 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d67f2a-b63b-4e22-be13-6652ca9b1cd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:01:43.236305 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d67f2a-b63b-4e22-be13-6652ca9b1cd1 admin admin] PATH_INFO: `/v3/projects/7ace18b7cfda4329adb560220ff909cc` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:01:43.237525 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d67f2a-b63b-4e22-be13-6652ca9b1cd1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ace18b7cfda4329adb560220ff909cc)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:01:43.239474 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d67f2a-b63b-4e22-be13-6652ca9b1cd1 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:01:43.342775 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-43d67f2a-b63b-4e22-be13-6652ca9b1cd1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ace18b7cfda4329adb560220ff909cc', 'request_id': 'req-43d67f2a-b63b-4e22-be13-6652ca9b1cd1'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 23:01:43.433931 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-43d67f2a-b63b-4e22-be13-6652ca9b1cd1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ace18b7cfda4329adb560220ff909cc', 'request_id': 'req-43d67f2a-b63b-4e22-be13-6652ca9b1cd1'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 23:01:43.439809 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-43d67f2a-b63b-4e22-be13-6652ca9b1cd1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ace18b7cfda4329adb560220ff909cc', 'request_id': 'req-43d67f2a-b63b-4e22-be13-6652ca9b1cd1'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 23:01:43.451031 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-43d67f2a-b63b-4e22-be13-6652ca9b1cd1 admin admin] The token cache is being invalidate because project 7ace18b7cfda4329adb560220ff909cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 23:01:43.451342 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-43d67f2a-b63b-4e22-be13-6652ca9b1cd1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 23:01:43.451982 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-43d67f2a-b63b-4e22-be13-6652ca9b1cd1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 23:01:43.453575 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1390/2784] 10.222.0.81 () {58 vars in 1232 bytes} [Wed Jul 20 23:01:43 2022] DELETE /identity/v3/projects/7ace18b7cfda4329adb560220ff909cc => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 23:01:43.459338 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f72a8c-37db-4d58-9ea8-474cd5e80a76 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:01:43.459842 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f72a8c-37db-4d58-9ea8-474cd5e80a76 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:01:43.459958 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f72a8c-37db-4d58-9ea8-474cd5e80a76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:01:43.478204 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-60f72a8c-37db-4d58-9ea8-474cd5e80a76 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 23:01:43.479469 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-60f72a8c-37db-4d58-9ea8-474cd5e80a76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 23:01:43.538289 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1395/2785] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 23:01:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 23:01:43.546239 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958ddd2e-e20f-4457-8a4c-0dc50cccd106 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:01:43.548131 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958ddd2e-e20f-4457-8a4c-0dc50cccd106 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:01:43.548131 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958ddd2e-e20f-4457-8a4c-0dc50cccd106 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:01:43.565359 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-958ddd2e-e20f-4457-8a4c-0dc50cccd106 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 23:01:43.566566 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-958ddd2e-e20f-4457-8a4c-0dc50cccd106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 23:01:43.574545 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1391/2786] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 23:01:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 23:01:43.593423 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f72a8c-37db-4d58-9ea8-474cd5e80a76 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:01:43.603752 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f72a8c-37db-4d58-9ea8-474cd5e80a76 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:01:43.608589 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201cf8ff-a8fe-477e-9d6d-1953d031d2ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:01:43.609260 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201cf8ff-a8fe-477e-9d6d-1953d031d2ea admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:01:43.609551 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201cf8ff-a8fe-477e-9d6d-1953d031d2ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:01:43.613574 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201cf8ff-a8fe-477e-9d6d-1953d031d2ea admin admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:01:43.613574 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201cf8ff-a8fe-477e-9d6d-1953d031d2ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 23:01:43.613574 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201cf8ff-a8fe-477e-9d6d-1953d031d2ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:01:43.613574 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201cf8ff-a8fe-477e-9d6d-1953d031d2ea admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:01:43.614172 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 23:01:43.624818 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1396/2787] 10.222.0.81 () {58 vars in 1152 bytes} [Wed Jul 20 23:01:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 23:01:43.629593 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958ddd2e-e20f-4457-8a4c-0dc50cccd106 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:01:43.639226 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958ddd2e-e20f-4457-8a4c-0dc50cccd106 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:01:43.644539 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5729f2-e56b-4ab3-8335-5598bdc6ed37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:01:43.645635 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5729f2-e56b-4ab3-8335-5598bdc6ed37 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:01:43.645635 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5729f2-e56b-4ab3-8335-5598bdc6ed37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:01:43.645854 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5729f2-e56b-4ab3-8335-5598bdc6ed37 admin admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:01:43.646413 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5729f2-e56b-4ab3-8335-5598bdc6ed37 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:01:43.648359 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5729f2-e56b-4ab3-8335-5598bdc6ed37 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:01:43.658148 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 23:01:43.737129 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1392/2788] 10.222.0.81 () {60 vars in 1152 bytes} [Wed Jul 20 23:01:43 2022] POST /identity/v3/projects => generated 357 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 23:01:43.741202 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201cf8ff-a8fe-477e-9d6d-1953d031d2ea admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:01:43.750996 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201cf8ff-a8fe-477e-9d6d-1953d031d2ea admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:01:43.806369 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dd8f28-3fda-402a-8d96-99d5bcc9a811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:01:43.807067 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dd8f28-3fda-402a-8d96-99d5bcc9a811 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:01:43.807362 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dd8f28-3fda-402a-8d96-99d5bcc9a811 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:01:43.807643 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dd8f28-3fda-402a-8d96-99d5bcc9a811 admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:01:43.808370 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dd8f28-3fda-402a-8d96-99d5bcc9a811 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:01:43.810272 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dd8f28-3fda-402a-8d96-99d5bcc9a811 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:01:43.936728 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1397/2789] 10.222.0.81 () {60 vars in 1143 bytes} [Wed Jul 20 23:01:43 2022] POST /identity/v3/users => generated 279 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 23:01:43.941306 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5729f2-e56b-4ab3-8335-5598bdc6ed37 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:01:43.952386 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5729f2-e56b-4ab3-8335-5598bdc6ed37 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:01:43.957695 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34826570-f2b3-462d-a368-e2957482bd3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:01:43.958362 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34826570-f2b3-462d-a368-e2957482bd3a admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:01:43.958652 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34826570-f2b3-462d-a368-e2957482bd3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:01:43.958936 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34826570-f2b3-462d-a368-e2957482bd3a admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:01:43.959703 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34826570-f2b3-462d-a368-e2957482bd3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:01:43.961312 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34826570-f2b3-462d-a368-e2957482bd3a admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:01:43.971292 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1393/2790] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 23:01:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 23:01:43.975106 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dd8f28-3fda-402a-8d96-99d5bcc9a811 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:01:43.985053 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dd8f28-3fda-402a-8d96-99d5bcc9a811 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:01:43.990293 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d4af83-6ed4-4bb1-a36f-edc608723af7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:01:43.990901 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d4af83-6ed4-4bb1-a36f-edc608723af7 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:01:43.991221 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d4af83-6ed4-4bb1-a36f-edc608723af7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:01:43.991507 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d4af83-6ed4-4bb1-a36f-edc608723af7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:01:43.992241 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d4af83-6ed4-4bb1-a36f-edc608723af7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:01:43.993832 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d4af83-6ed4-4bb1-a36f-edc608723af7 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:01:44.004273 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1398/2791] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 23:01:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 23:01:44.010039 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34826570-f2b3-462d-a368-e2957482bd3a admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:01:44.020350 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34826570-f2b3-462d-a368-e2957482bd3a admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:01:44.024914 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b760776-e7ef-404f-ba8f-e155378e5d8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:01:44.025792 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b760776-e7ef-404f-ba8f-e155378e5d8b admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:01:44.026079 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b760776-e7ef-404f-ba8f-e155378e5d8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:01:44.026366 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b760776-e7ef-404f-ba8f-e155378e5d8b admin admin] PATH_INFO: `/v3/projects/778e0a94fb22449892fb7af416b9ee89/users/111cb9f8911c452ca1063abb9c989b01/roles/559942c4de1744169f8b5e0404ecde6b` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:01:44.042423 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b760776-e7ef-404f-ba8f-e155378e5d8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=778e0a94fb22449892fb7af416b9ee89, user_id=111cb9f8911c452ca1063abb9c989b01, role_id=559942c4de1744169f8b5e0404ecde6b)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:01:44.045100 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b760776-e7ef-404f-ba8f-e155378e5d8b admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:01:44.161249 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1394/2792] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 23:01:44 2022] PUT /identity/v3/projects/778e0a94fb22449892fb7af416b9ee89/users/111cb9f8911c452ca1063abb9c989b01/roles/559942c4de1744169f8b5e0404ecde6b => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 23:01:44.172438 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad09f4f4-5b93-41cb-b1fa-f4bcc637f1fa None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:01:44.172735 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad09f4f4-5b93-41cb-b1fa-f4bcc637f1fa None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:01:44.173023 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad09f4f4-5b93-41cb-b1fa-f4bcc637f1fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:01:44.199646 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-ad09f4f4-5b93-41cb-b1fa-f4bcc637f1fa None None] MFA Rules not processed for user `111cb9f8911c452ca1063abb9c989b01`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 23:01:44.200864 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-ad09f4f4-5b93-41cb-b1fa-f4bcc637f1fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 23:01:44.264783 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1399/2793] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 23:01:44 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 23:01:44.269753 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b760776-e7ef-404f-ba8f-e155378e5d8b admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:01:44.331216 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b760776-e7ef-404f-ba8f-e155378e5d8b admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:01:44.335854 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd115fe-9c20-4122-8928-3e4f0843dc57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:01:44.336511 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd115fe-9c20-4122-8928-3e4f0843dc57 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:01:44.336796 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd115fe-9c20-4122-8928-3e4f0843dc57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:01:44.337077 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd115fe-9c20-4122-8928-3e4f0843dc57 admin admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:01:44.337756 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd115fe-9c20-4122-8928-3e4f0843dc57 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:01:44.339699 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd115fe-9c20-4122-8928-3e4f0843dc57 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:01:44.425404 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1395/2794] 10.222.0.81 () {60 vars in 1152 bytes} [Wed Jul 20 23:01:44 2022] POST /identity/v3/projects => generated 355 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 23:01:44.437849 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad09f4f4-5b93-41cb-b1fa-f4bcc637f1fa None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:01:44.449307 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad09f4f4-5b93-41cb-b1fa-f4bcc637f1fa None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:01:44.505027 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbbed0a-8873-4b31-b610-474290029e56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:01:44.505706 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbbed0a-8873-4b31-b610-474290029e56 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:01:44.505996 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbbed0a-8873-4b31-b610-474290029e56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:01:44.506301 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbbed0a-8873-4b31-b610-474290029e56 admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:01:44.507001 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbbed0a-8873-4b31-b610-474290029e56 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:01:44.509339 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbbed0a-8873-4b31-b610-474290029e56 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:01:44.517437 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 23:01:44.669856 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1400/2795] 10.222.0.81 () {60 vars in 1143 bytes} [Wed Jul 20 23:01:44 2022] POST /identity/v3/users => generated 284 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 23:01:44.673848 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd115fe-9c20-4122-8928-3e4f0843dc57 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:01:44.684323 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd115fe-9c20-4122-8928-3e4f0843dc57 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:01:44.689394 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b02cd90-323b-445b-85c1-fabe86500ece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:01:44.690057 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b02cd90-323b-445b-85c1-fabe86500ece admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:01:44.690345 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b02cd90-323b-445b-85c1-fabe86500ece admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:01:44.690642 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b02cd90-323b-445b-85c1-fabe86500ece admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:01:44.691390 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b02cd90-323b-445b-85c1-fabe86500ece admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:01:44.692985 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b02cd90-323b-445b-85c1-fabe86500ece admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:01:44.703439 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1396/2796] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 23:01:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 23:01:44.707959 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbbed0a-8873-4b31-b610-474290029e56 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:01:44.718165 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbbed0a-8873-4b31-b610-474290029e56 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:01:44.723340 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de635545-17e9-42cc-a3fb-371bc1c6dab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:01:44.724270 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de635545-17e9-42cc-a3fb-371bc1c6dab1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:01:44.724557 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de635545-17e9-42cc-a3fb-371bc1c6dab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:01:44.724837 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de635545-17e9-42cc-a3fb-371bc1c6dab1 admin admin] PATH_INFO: `/v3/projects/65aa6a128d37468a9259183321c113fe/users/ffb76e8e88c74fcf9003b8787b402bf7/roles/d89b6c92d35a401a8517f1d47f6be843` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:01:44.741939 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de635545-17e9-42cc-a3fb-371bc1c6dab1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65aa6a128d37468a9259183321c113fe, user_id=ffb76e8e88c74fcf9003b8787b402bf7, role_id=d89b6c92d35a401a8517f1d47f6be843)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:01:44.744770 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de635545-17e9-42cc-a3fb-371bc1c6dab1 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:01:44.858701 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1401/2797] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 23:01:44 2022] PUT /identity/v3/projects/65aa6a128d37468a9259183321c113fe/users/ffb76e8e88c74fcf9003b8787b402bf7/roles/d89b6c92d35a401a8517f1d47f6be843 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 23:01:44.863413 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b02cd90-323b-445b-85c1-fabe86500ece admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:01:44.873598 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b02cd90-323b-445b-85c1-fabe86500ece admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:01:44.928795 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6e700c-68ae-429d-b6f7-7a230b7796b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:01:44.929464 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6e700c-68ae-429d-b6f7-7a230b7796b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:01:44.929753 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6e700c-68ae-429d-b6f7-7a230b7796b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:01:44.930038 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6e700c-68ae-429d-b6f7-7a230b7796b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:01:44.930762 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6e700c-68ae-429d-b6f7-7a230b7796b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:01:44.932412 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6e700c-68ae-429d-b6f7-7a230b7796b1 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:01:44.942598 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1397/2798] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 23:01:44 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 23:01:44.946913 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de635545-17e9-42cc-a3fb-371bc1c6dab1 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:01:44.957194 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de635545-17e9-42cc-a3fb-371bc1c6dab1 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:01:44.961893 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfd3d6e-eeef-4ae8-9085-807602f6540c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:01:44.962777 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfd3d6e-eeef-4ae8-9085-807602f6540c admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:01:44.963095 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfd3d6e-eeef-4ae8-9085-807602f6540c admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:01:44.963374 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfd3d6e-eeef-4ae8-9085-807602f6540c admin admin] PATH_INFO: `/v3/projects/65aa6a128d37468a9259183321c113fe/users/ffb76e8e88c74fcf9003b8787b402bf7/roles/d89b6c92d35a401a8517f1d47f6be843` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:01:44.965344 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfd3d6e-eeef-4ae8-9085-807602f6540c admin admin] RBAC: Authorizing `identity:create_grant(project_id=65aa6a128d37468a9259183321c113fe, user_id=ffb76e8e88c74fcf9003b8787b402bf7, role_id=d89b6c92d35a401a8517f1d47f6be843)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:01:44.967966 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfd3d6e-eeef-4ae8-9085-807602f6540c admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:01:44.977901 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1402/2799] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 23:01:44 2022] PUT /identity/v3/projects/65aa6a128d37468a9259183321c113fe/users/ffb76e8e88c74fcf9003b8787b402bf7/roles/d89b6c92d35a401a8517f1d47f6be843 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 23:01:44.988348 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe1f641-ad6d-49e6-96a8-e7a9c31a61e5 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:01:44.988646 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe1f641-ad6d-49e6-96a8-e7a9c31a61e5 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:01:44.988929 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe1f641-ad6d-49e6-96a8-e7a9c31a61e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:01:45.015447 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-ffe1f641-ad6d-49e6-96a8-e7a9c31a61e5 None None] MFA Rules not processed for user `ffb76e8e88c74fcf9003b8787b402bf7`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 23:01:45.016667 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-ffe1f641-ad6d-49e6-96a8-e7a9c31a61e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 23:01:45.029045 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 23:01:45.085551 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1398/2800] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 23:01:44 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 23:01:45.097434 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfd3d6e-eeef-4ae8-9085-807602f6540c admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:01:45.099151 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-bdfd3d6e-eeef-4ae8-9085-807602f6540c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 23:01:45.278344 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfd3d6e-eeef-4ae8-9085-807602f6540c admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:01:45.282040 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2e6943-dc2c-41ab-a7ee-1d8cac68c0b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecf4db289fba4e4a806198c62d31f7c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:01:45.283241 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2e6943-dc2c-41ab-a7ee-1d8cac68c0b4 service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:01:45.283550 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2e6943-dc2c-41ab-a7ee-1d8cac68c0b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:01:45.283828 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2e6943-dc2c-41ab-a7ee-1d8cac68c0b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:01:45.292721 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2e6943-dc2c-41ab-a7ee-1d8cac68c0b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:01:45.294125 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2e6943-dc2c-41ab-a7ee-1d8cac68c0b4 service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:01:45.304135 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1403/2801] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 23:01:45 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 23:01:45.549860 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe1f641-ad6d-49e6-96a8-e7a9c31a61e5 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:01:45.559650 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe1f641-ad6d-49e6-96a8-e7a9c31a61e5 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:01:45.563500 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b10e65-36ff-44a6-aca2-d172e2aa842b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecf4db289fba4e4a806198c62d31f7c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:01:45.564186 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b10e65-36ff-44a6-aca2-d172e2aa842b service cinder] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:01:45.564473 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b10e65-36ff-44a6-aca2-d172e2aa842b service cinder] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:01:45.564757 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b10e65-36ff-44a6-aca2-d172e2aa842b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:01:45.621040 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b10e65-36ff-44a6-aca2-d172e2aa842b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:01:45.622480 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b10e65-36ff-44a6-aca2-d172e2aa842b service cinder] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:01:45.631601 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1399/2802] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 23:01:45 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 23:01:55.299874 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:INSERT=4,keystone:SELECT=43 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 23:01:55.627306 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=50 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 23:02:04.864016 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2e6943-dc2c-41ab-a7ee-1d8cac68c0b4 service cinder] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:02:04.925105 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2e6943-dc2c-41ab-a7ee-1d8cac68c0b4 service cinder] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:02:04.929892 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aec0c1-0a1b-4ef5-b039-f8d5bcecc3ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:02:04.930810 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aec0c1-0a1b-4ef5-b039-f8d5bcecc3ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:02:04.931124 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aec0c1-0a1b-4ef5-b039-f8d5bcecc3ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:02:04.931405 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aec0c1-0a1b-4ef5-b039-f8d5bcecc3ef admin admin] PATH_INFO: `/v3/users/111cb9f8911c452ca1063abb9c989b01` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:02:04.932633 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82aec0c1-0a1b-4ef5-b039-f8d5bcecc3ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=111cb9f8911c452ca1063abb9c989b01)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:02:04.934560 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82aec0c1-0a1b-4ef5-b039-f8d5bcecc3ef admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:02:05.092673 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-82aec0c1-0a1b-4ef5-b039-f8d5bcecc3ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '111cb9f8911c452ca1063abb9c989b01'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 23:02:05.093342 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-82aec0c1-0a1b-4ef5-b039-f8d5bcecc3ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '111cb9f8911c452ca1063abb9c989b01'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 23:02:05.093944 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-82aec0c1-0a1b-4ef5-b039-f8d5bcecc3ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '111cb9f8911c452ca1063abb9c989b01'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 23:02:05.101024 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-82aec0c1-0a1b-4ef5-b039-f8d5bcecc3ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '111cb9f8911c452ca1063abb9c989b01'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 23:02:05.164681 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-82aec0c1-0a1b-4ef5-b039-f8d5bcecc3ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '111cb9f8911c452ca1063abb9c989b01'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 23:02:05.182179 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1404/2803] 10.222.0.81 () {58 vars in 1223 bytes} [Wed Jul 20 23:02:04 2022] DELETE /identity/v3/users/111cb9f8911c452ca1063abb9c989b01 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 23:02:05.186738 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3a084d-595e-409f-8472-9f352a6fbdbd None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:02:05.187260 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3a084d-595e-409f-8472-9f352a6fbdbd None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:02:05.187386 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3a084d-595e-409f-8472-9f352a6fbdbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:02:05.205891 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-5a3a084d-595e-409f-8472-9f352a6fbdbd None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 23:02:05.207169 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-5a3a084d-595e-409f-8472-9f352a6fbdbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 23:02:05.265057 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1400/2804] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 23:02:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 23:02:05.275359 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aec0c1-0a1b-4ef5-b039-f8d5bcecc3ef admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:02:05.276771 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-82aec0c1-0a1b-4ef5-b039-f8d5bcecc3ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 23:02:05.333887 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aec0c1-0a1b-4ef5-b039-f8d5bcecc3ef admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:02:05.337614 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7ce2cd-8757-40bb-9f68-8b634a487a3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:02:05.338258 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7ce2cd-8757-40bb-9f68-8b634a487a3a service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:02:05.338538 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7ce2cd-8757-40bb-9f68-8b634a487a3a service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:02:05.338813 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7ce2cd-8757-40bb-9f68-8b634a487a3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:02:05.348444 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7ce2cd-8757-40bb-9f68-8b634a487a3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:02:05.349860 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7ce2cd-8757-40bb-9f68-8b634a487a3a service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:02:05.359440 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1405/2805] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 23:02:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 23:02:05.795113 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3a084d-595e-409f-8472-9f352a6fbdbd None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:02:05.796571 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-5a3a084d-595e-409f-8472-9f352a6fbdbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 23:02:05.810877 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3a084d-595e-409f-8472-9f352a6fbdbd None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:02:05.816217 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960a5f1c-2f8d-4f0a-87f0-ee39286a60c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:02:05.817114 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960a5f1c-2f8d-4f0a-87f0-ee39286a60c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:02:05.817402 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960a5f1c-2f8d-4f0a-87f0-ee39286a60c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:02:05.818222 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960a5f1c-2f8d-4f0a-87f0-ee39286a60c0 admin admin] PATH_INFO: `/v3/projects/778e0a94fb22449892fb7af416b9ee89` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:02:05.819752 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960a5f1c-2f8d-4f0a-87f0-ee39286a60c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=778e0a94fb22449892fb7af416b9ee89)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:02:05.821670 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960a5f1c-2f8d-4f0a-87f0-ee39286a60c0 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:02:05.925731 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-960a5f1c-2f8d-4f0a-87f0-ee39286a60c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '778e0a94fb22449892fb7af416b9ee89', 'request_id': 'req-960a5f1c-2f8d-4f0a-87f0-ee39286a60c0'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 23:02:05.999609 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-960a5f1c-2f8d-4f0a-87f0-ee39286a60c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '778e0a94fb22449892fb7af416b9ee89', 'request_id': 'req-960a5f1c-2f8d-4f0a-87f0-ee39286a60c0'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 23:02:06.006311 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-960a5f1c-2f8d-4f0a-87f0-ee39286a60c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '778e0a94fb22449892fb7af416b9ee89', 'request_id': 'req-960a5f1c-2f8d-4f0a-87f0-ee39286a60c0'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 23:02:06.017634 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-960a5f1c-2f8d-4f0a-87f0-ee39286a60c0 admin admin] The token cache is being invalidate because project 778e0a94fb22449892fb7af416b9ee89 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 23:02:06.017926 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-960a5f1c-2f8d-4f0a-87f0-ee39286a60c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 23:02:06.018561 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-960a5f1c-2f8d-4f0a-87f0-ee39286a60c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 23:02:06.020276 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1401/2806] 10.222.0.81 () {58 vars in 1232 bytes} [Wed Jul 20 23:02:05 2022] DELETE /identity/v3/projects/778e0a94fb22449892fb7af416b9ee89 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 23:02:06.024266 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7ce2cd-8757-40bb-9f68-8b634a487a3a service neutron] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:02:06.033784 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7ce2cd-8757-40bb-9f68-8b634a487a3a service neutron] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:02:06.052219 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 23:02:06.095057 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751f2b98-7be5-43d5-b356-c6084b9387ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:02:06.095956 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751f2b98-7be5-43d5-b356-c6084b9387ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:02:06.096240 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751f2b98-7be5-43d5-b356-c6084b9387ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:02:06.096521 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751f2b98-7be5-43d5-b356-c6084b9387ad admin admin] PATH_INFO: `/v3/users/ffb76e8e88c74fcf9003b8787b402bf7` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:02:06.097717 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751f2b98-7be5-43d5-b356-c6084b9387ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=ffb76e8e88c74fcf9003b8787b402bf7)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:02:06.099672 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751f2b98-7be5-43d5-b356-c6084b9387ad admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:02:06.321626 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-751f2b98-7be5-43d5-b356-c6084b9387ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ffb76e8e88c74fcf9003b8787b402bf7'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 23:02:06.322287 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-751f2b98-7be5-43d5-b356-c6084b9387ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ffb76e8e88c74fcf9003b8787b402bf7'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 23:02:06.322917 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-751f2b98-7be5-43d5-b356-c6084b9387ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ffb76e8e88c74fcf9003b8787b402bf7'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 23:02:06.329763 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-751f2b98-7be5-43d5-b356-c6084b9387ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ffb76e8e88c74fcf9003b8787b402bf7'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 23:02:06.416795 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-751f2b98-7be5-43d5-b356-c6084b9387ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ffb76e8e88c74fcf9003b8787b402bf7'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 23:02:06.435207 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1406/2807] 10.222.0.81 () {58 vars in 1223 bytes} [Wed Jul 20 23:02:06 2022] DELETE /identity/v3/users/ffb76e8e88c74fcf9003b8787b402bf7 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 23:02:06.875250 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960a5f1c-2f8d-4f0a-87f0-ee39286a60c0 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:02:06.877233 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-960a5f1c-2f8d-4f0a-87f0-ee39286a60c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 23:02:06.891109 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960a5f1c-2f8d-4f0a-87f0-ee39286a60c0 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:02:06.949497 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221ee8ad-966c-4723-a01a-7965beb31686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:02:06.950403 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221ee8ad-966c-4723-a01a-7965beb31686 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:02:06.950687 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221ee8ad-966c-4723-a01a-7965beb31686 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:02:06.950962 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221ee8ad-966c-4723-a01a-7965beb31686 admin admin] PATH_INFO: `/v3/projects/65aa6a128d37468a9259183321c113fe` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:02:06.952219 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221ee8ad-966c-4723-a01a-7965beb31686 admin admin] RBAC: Authorizing `identity:delete_project(project_id=65aa6a128d37468a9259183321c113fe)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:02:06.954808 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221ee8ad-966c-4723-a01a-7965beb31686 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:02:07.109903 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 23:02:07.127413 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-221ee8ad-966c-4723-a01a-7965beb31686 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65aa6a128d37468a9259183321c113fe', 'request_id': 'req-221ee8ad-966c-4723-a01a-7965beb31686'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 23:02:07.238953 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-221ee8ad-966c-4723-a01a-7965beb31686 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65aa6a128d37468a9259183321c113fe', 'request_id': 'req-221ee8ad-966c-4723-a01a-7965beb31686'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 23:02:07.245069 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-221ee8ad-966c-4723-a01a-7965beb31686 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65aa6a128d37468a9259183321c113fe', 'request_id': 'req-221ee8ad-966c-4723-a01a-7965beb31686'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 23:02:07.256670 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-221ee8ad-966c-4723-a01a-7965beb31686 admin admin] The token cache is being invalidate because project 65aa6a128d37468a9259183321c113fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 23:02:07.256965 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-221ee8ad-966c-4723-a01a-7965beb31686 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 23:02:07.257594 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-221ee8ad-966c-4723-a01a-7965beb31686 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 23:02:07.259430 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1402/2808] 10.222.0.81 () {58 vars in 1232 bytes} [Wed Jul 20 23:02:06 2022] DELETE /identity/v3/projects/65aa6a128d37468a9259183321c113fe => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 23:02:07.265208 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5a190f-3909-4e4e-95f0-4c05d69d39c5 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:02:07.265688 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5a190f-3909-4e4e-95f0-4c05d69d39c5 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:02:07.265807 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5a190f-3909-4e4e-95f0-4c05d69d39c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:02:07.294958 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-ba5a190f-3909-4e4e-95f0-4c05d69d39c5 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 23:02:07.296230 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-ba5a190f-3909-4e4e-95f0-4c05d69d39c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 23:02:07.356928 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1407/2809] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 23:02:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 23:02:07.365114 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf2b814-aa9b-49ee-80e3-a1cfde88056a None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:02:07.365406 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf2b814-aa9b-49ee-80e3-a1cfde88056a None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:02:07.365689 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf2b814-aa9b-49ee-80e3-a1cfde88056a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:02:07.385581 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-7bf2b814-aa9b-49ee-80e3-a1cfde88056a None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 23:02:07.386797 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-7bf2b814-aa9b-49ee-80e3-a1cfde88056a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 23:02:07.394706 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1403/2810] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 23:02:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 23:02:07.398690 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5a190f-3909-4e4e-95f0-4c05d69d39c5 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:02:07.408109 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5a190f-3909-4e4e-95f0-4c05d69d39c5 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:02:07.412729 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41dfaf5-8971-43c5-81f2-7d1d11734bba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:02:07.413383 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41dfaf5-8971-43c5-81f2-7d1d11734bba admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:02:07.413665 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41dfaf5-8971-43c5-81f2-7d1d11734bba admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:02:07.413938 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41dfaf5-8971-43c5-81f2-7d1d11734bba admin admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:02:07.414685 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41dfaf5-8971-43c5-81f2-7d1d11734bba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 23:02:07.414999 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41dfaf5-8971-43c5-81f2-7d1d11734bba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:02:07.416599 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41dfaf5-8971-43c5-81f2-7d1d11734bba admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:02:07.428583 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1408/2811] 10.222.0.81 () {58 vars in 1152 bytes} [Wed Jul 20 23:02:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 23:02:07.432709 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf2b814-aa9b-49ee-80e3-a1cfde88056a None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:02:07.442571 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf2b814-aa9b-49ee-80e3-a1cfde88056a None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:02:07.447234 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587a1745-b9f5-4be4-8e2c-ef2efd45abdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:02:07.447883 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587a1745-b9f5-4be4-8e2c-ef2efd45abdc admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:02:07.448171 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587a1745-b9f5-4be4-8e2c-ef2efd45abdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:02:07.448444 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587a1745-b9f5-4be4-8e2c-ef2efd45abdc admin admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:02:07.449155 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587a1745-b9f5-4be4-8e2c-ef2efd45abdc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:02:07.451057 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587a1745-b9f5-4be4-8e2c-ef2efd45abdc admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:02:07.600636 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1404/2812] 10.222.0.81 () {60 vars in 1152 bytes} [Wed Jul 20 23:02:07 2022] POST /identity/v3/projects => generated 387 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 23:02:07.604829 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41dfaf5-8971-43c5-81f2-7d1d11734bba admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:02:07.625723 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41dfaf5-8971-43c5-81f2-7d1d11734bba admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:02:07.663103 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 23:02:07.692536 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0efe82-8ab7-4a5d-bf88-d86a04761775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:02:07.693214 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0efe82-8ab7-4a5d-bf88-d86a04761775 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:02:07.693499 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0efe82-8ab7-4a5d-bf88-d86a04761775 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:02:07.693773 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0efe82-8ab7-4a5d-bf88-d86a04761775 admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:02:07.694449 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0efe82-8ab7-4a5d-bf88-d86a04761775 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:02:07.696371 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0efe82-8ab7-4a5d-bf88-d86a04761775 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:02:07.844012 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1409/2813] 10.222.0.81 () {60 vars in 1143 bytes} [Wed Jul 20 23:02:07 2022] POST /identity/v3/users => generated 294 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 23:02:07.848309 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587a1745-b9f5-4be4-8e2c-ef2efd45abdc admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:02:07.858808 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587a1745-b9f5-4be4-8e2c-ef2efd45abdc admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:02:07.863443 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d200548-4bdb-41d7-be1c-ca7f2da70f0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:02:07.864108 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d200548-4bdb-41d7-be1c-ca7f2da70f0e admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:02:07.864400 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d200548-4bdb-41d7-be1c-ca7f2da70f0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:02:07.864678 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d200548-4bdb-41d7-be1c-ca7f2da70f0e admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:02:07.865459 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d200548-4bdb-41d7-be1c-ca7f2da70f0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:02:07.867074 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d200548-4bdb-41d7-be1c-ca7f2da70f0e admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:02:07.878131 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1405/2814] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 23:02:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 23:02:07.882011 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0efe82-8ab7-4a5d-bf88-d86a04761775 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:02:07.893347 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0efe82-8ab7-4a5d-bf88-d86a04761775 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:02:07.898155 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365df2b8-cf39-4823-9d80-b13952bef0f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:02:07.898834 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365df2b8-cf39-4823-9d80-b13952bef0f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:02:07.899160 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365df2b8-cf39-4823-9d80-b13952bef0f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:02:07.899445 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365df2b8-cf39-4823-9d80-b13952bef0f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:02:07.900179 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365df2b8-cf39-4823-9d80-b13952bef0f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:02:07.901771 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365df2b8-cf39-4823-9d80-b13952bef0f4 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:02:07.912163 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1410/2815] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 23:02:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 23:02:07.916425 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d200548-4bdb-41d7-be1c-ca7f2da70f0e admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:02:07.926387 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d200548-4bdb-41d7-be1c-ca7f2da70f0e admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:02:07.930900 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d89d95-b137-422c-9be5-3f6d9eb14abd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:02:07.931813 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d89d95-b137-422c-9be5-3f6d9eb14abd admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:02:07.932098 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d89d95-b137-422c-9be5-3f6d9eb14abd admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:02:07.932402 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d89d95-b137-422c-9be5-3f6d9eb14abd admin admin] PATH_INFO: `/v3/projects/8a9b7dd3df944818b1f5315ff3e07011/users/4d7fa518faf34eb29f234aea7b6a94d3/roles/559942c4de1744169f8b5e0404ecde6b` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:02:07.947279 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d89d95-b137-422c-9be5-3f6d9eb14abd admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a9b7dd3df944818b1f5315ff3e07011, user_id=4d7fa518faf34eb29f234aea7b6a94d3, role_id=559942c4de1744169f8b5e0404ecde6b)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:02:07.949934 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d89d95-b137-422c-9be5-3f6d9eb14abd admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:02:08.056638 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1406/2816] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 23:02:07 2022] PUT /identity/v3/projects/8a9b7dd3df944818b1f5315ff3e07011/users/4d7fa518faf34eb29f234aea7b6a94d3/roles/559942c4de1744169f8b5e0404ecde6b => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 23:02:08.068485 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9607d56-e239-41d1-a40a-76103d5a8a9f None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:02:08.068788 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9607d56-e239-41d1-a40a-76103d5a8a9f None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:02:08.069057 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9607d56-e239-41d1-a40a-76103d5a8a9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:02:08.095511 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-d9607d56-e239-41d1-a40a-76103d5a8a9f None None] MFA Rules not processed for user `4d7fa518faf34eb29f234aea7b6a94d3`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 23:02:08.096720 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-d9607d56-e239-41d1-a40a-76103d5a8a9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 23:02:08.160520 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1411/2817] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 23:02:08 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 23:02:08.164950 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d89d95-b137-422c-9be5-3f6d9eb14abd admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:02:08.228720 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d89d95-b137-422c-9be5-3f6d9eb14abd admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:02:08.233514 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7684ac8-3e92-495e-ac2b-fe017dede5e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:02:08.234176 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7684ac8-3e92-495e-ac2b-fe017dede5e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:02:08.234465 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7684ac8-3e92-495e-ac2b-fe017dede5e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:02:08.234749 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7684ac8-3e92-495e-ac2b-fe017dede5e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:02:08.235978 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7684ac8-3e92-495e-ac2b-fe017dede5e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:02:08.237890 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7684ac8-3e92-495e-ac2b-fe017dede5e7 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:02:08.354800 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1407/2818] 10.222.0.81 () {60 vars in 1152 bytes} [Wed Jul 20 23:02:08 2022] POST /identity/v3/projects => generated 387 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 23:02:08.358953 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9607d56-e239-41d1-a40a-76103d5a8a9f None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:02:08.371101 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9607d56-e239-41d1-a40a-76103d5a8a9f None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:02:08.426656 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc77d047-ddbc-457e-b0eb-73934ff29174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:02:08.427300 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc77d047-ddbc-457e-b0eb-73934ff29174 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:02:08.427588 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc77d047-ddbc-457e-b0eb-73934ff29174 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:02:08.427867 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc77d047-ddbc-457e-b0eb-73934ff29174 admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:02:08.428558 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc77d047-ddbc-457e-b0eb-73934ff29174 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:02:08.430456 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc77d047-ddbc-457e-b0eb-73934ff29174 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:02:08.495987 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1412/2819] 10.222.0.81 () {60 vars in 1143 bytes} [Wed Jul 20 23:02:08 2022] POST /identity/v3/users => generated 300 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 23:02:08.500066 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7684ac8-3e92-495e-ac2b-fe017dede5e7 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:02:08.510534 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7684ac8-3e92-495e-ac2b-fe017dede5e7 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:02:08.515280 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e0df93-a292-44de-89de-46ad2fd00f1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:02:08.515944 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e0df93-a292-44de-89de-46ad2fd00f1b admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:02:08.516263 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e0df93-a292-44de-89de-46ad2fd00f1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:02:08.516548 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e0df93-a292-44de-89de-46ad2fd00f1b admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:02:08.517277 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e0df93-a292-44de-89de-46ad2fd00f1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:02:08.518881 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e0df93-a292-44de-89de-46ad2fd00f1b admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:02:08.529400 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1408/2820] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 23:02:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 23:02:08.533789 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc77d047-ddbc-457e-b0eb-73934ff29174 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:02:08.543561 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc77d047-ddbc-457e-b0eb-73934ff29174 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:02:08.548818 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12650bd4-a017-4181-b9d6-aa70f60a5c7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:02:08.549702 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12650bd4-a017-4181-b9d6-aa70f60a5c7a admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:02:08.549988 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12650bd4-a017-4181-b9d6-aa70f60a5c7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:02:08.550263 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12650bd4-a017-4181-b9d6-aa70f60a5c7a admin admin] PATH_INFO: `/v3/projects/30872aaaa4ce440cb3ee682da6d03e6f/users/32fcc71d27874a588ad61d617dd3cf2c/roles/d89b6c92d35a401a8517f1d47f6be843` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:02:08.564485 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12650bd4-a017-4181-b9d6-aa70f60a5c7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=30872aaaa4ce440cb3ee682da6d03e6f, user_id=32fcc71d27874a588ad61d617dd3cf2c, role_id=d89b6c92d35a401a8517f1d47f6be843)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:02:08.567132 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12650bd4-a017-4181-b9d6-aa70f60a5c7a admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:02:08.682704 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1413/2821] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 23:02:08 2022] PUT /identity/v3/projects/30872aaaa4ce440cb3ee682da6d03e6f/users/32fcc71d27874a588ad61d617dd3cf2c/roles/d89b6c92d35a401a8517f1d47f6be843 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 23:02:08.687104 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e0df93-a292-44de-89de-46ad2fd00f1b admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:02:08.699934 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e0df93-a292-44de-89de-46ad2fd00f1b admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:02:08.720100 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 23:02:08.755951 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1193bd-2403-43cc-a905-0d6ca85dee07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:02:08.756625 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1193bd-2403-43cc-a905-0d6ca85dee07 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:02:08.756946 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1193bd-2403-43cc-a905-0d6ca85dee07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:02:08.757230 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1193bd-2403-43cc-a905-0d6ca85dee07 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:02:08.757995 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1193bd-2403-43cc-a905-0d6ca85dee07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:02:08.760462 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1193bd-2403-43cc-a905-0d6ca85dee07 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:02:08.770876 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1409/2822] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 23:02:08 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 23:02:08.775198 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12650bd4-a017-4181-b9d6-aa70f60a5c7a admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:02:08.780588 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 23:02:08.786608 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12650bd4-a017-4181-b9d6-aa70f60a5c7a admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:02:08.791518 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db968e5-8497-41b1-9998-ffabe7b748f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:02:08.792487 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db968e5-8497-41b1-9998-ffabe7b748f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:02:08.792754 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db968e5-8497-41b1-9998-ffabe7b748f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:02:08.793026 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db968e5-8497-41b1-9998-ffabe7b748f1 admin admin] PATH_INFO: `/v3/projects/30872aaaa4ce440cb3ee682da6d03e6f/users/32fcc71d27874a588ad61d617dd3cf2c/roles/d89b6c92d35a401a8517f1d47f6be843` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:02:08.795148 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db968e5-8497-41b1-9998-ffabe7b748f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=30872aaaa4ce440cb3ee682da6d03e6f, user_id=32fcc71d27874a588ad61d617dd3cf2c, role_id=d89b6c92d35a401a8517f1d47f6be843)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:02:08.797773 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db968e5-8497-41b1-9998-ffabe7b748f1 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:02:08.807943 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1414/2823] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 23:02:08 2022] PUT /identity/v3/projects/30872aaaa4ce440cb3ee682da6d03e6f/users/32fcc71d27874a588ad61d617dd3cf2c/roles/d89b6c92d35a401a8517f1d47f6be843 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 23:02:08.818201 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59dfb19-164c-42c8-9c57-d5c3feadc022 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:02:08.818495 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59dfb19-164c-42c8-9c57-d5c3feadc022 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:02:08.818778 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59dfb19-164c-42c8-9c57-d5c3feadc022 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:02:08.845681 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-f59dfb19-164c-42c8-9c57-d5c3feadc022 None None] MFA Rules not processed for user `32fcc71d27874a588ad61d617dd3cf2c`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 23:02:08.846897 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-f59dfb19-164c-42c8-9c57-d5c3feadc022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 23:02:08.914531 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1410/2824] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 23:02:08 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 23:02:08.925904 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db968e5-8497-41b1-9998-ffabe7b748f1 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:02:08.927631 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-4db968e5-8497-41b1-9998-ffabe7b748f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 23:02:08.984101 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db968e5-8497-41b1-9998-ffabe7b748f1 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:02:08.987606 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a271efd9-0dd7-4da7-9b86-8e17364b6514 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecf4db289fba4e4a806198c62d31f7c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:02:08.988248 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a271efd9-0dd7-4da7-9b86-8e17364b6514 service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:02:08.988530 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a271efd9-0dd7-4da7-9b86-8e17364b6514 service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:02:08.988807 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a271efd9-0dd7-4da7-9b86-8e17364b6514 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:02:08.997471 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a271efd9-0dd7-4da7-9b86-8e17364b6514 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:02:08.998923 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a271efd9-0dd7-4da7-9b86-8e17364b6514 service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:02:09.008135 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1415/2825] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 23:02:08 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 23:02:09.036736 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59dfb19-164c-42c8-9c57-d5c3feadc022 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:02:09.097645 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59dfb19-164c-42c8-9c57-d5c3feadc022 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:02:09.102434 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fa4307-9960-4456-9df9-6e8f5336de77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:02:09.103354 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fa4307-9960-4456-9df9-6e8f5336de77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:02:09.103646 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fa4307-9960-4456-9df9-6e8f5336de77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:02:09.103928 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fa4307-9960-4456-9df9-6e8f5336de77 admin admin] PATH_INFO: `/v3/users/4d7fa518faf34eb29f234aea7b6a94d3` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:02:09.105138 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fa4307-9960-4456-9df9-6e8f5336de77 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d7fa518faf34eb29f234aea7b6a94d3)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:02:09.107093 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fa4307-9960-4456-9df9-6e8f5336de77 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:02:09.281953 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-e3fa4307-9960-4456-9df9-6e8f5336de77 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d7fa518faf34eb29f234aea7b6a94d3'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 23:02:09.282535 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-e3fa4307-9960-4456-9df9-6e8f5336de77 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d7fa518faf34eb29f234aea7b6a94d3'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 23:02:09.283440 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-e3fa4307-9960-4456-9df9-6e8f5336de77 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d7fa518faf34eb29f234aea7b6a94d3'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 23:02:09.284685 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 23:02:09.290945 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-e3fa4307-9960-4456-9df9-6e8f5336de77 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d7fa518faf34eb29f234aea7b6a94d3'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 23:02:09.398788 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-e3fa4307-9960-4456-9df9-6e8f5336de77 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d7fa518faf34eb29f234aea7b6a94d3'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 23:02:09.416172 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1411/2826] 10.222.0.81 () {58 vars in 1223 bytes} [Wed Jul 20 23:02:09 2022] DELETE /identity/v3/users/4d7fa518faf34eb29f234aea7b6a94d3 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 23:02:09.420736 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b82d39-fb14-4590-a465-8a4e259b2fca None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:02:09.421202 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b82d39-fb14-4590-a465-8a4e259b2fca None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:02:09.426140 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b82d39-fb14-4590-a465-8a4e259b2fca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:02:09.438613 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-b8b82d39-fb14-4590-a465-8a4e259b2fca None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 23:02:09.440014 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-b8b82d39-fb14-4590-a465-8a4e259b2fca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 23:02:09.497266 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1416/2827] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 23:02:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 23:02:09.507605 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fa4307-9960-4456-9df9-6e8f5336de77 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:02:09.512327 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-e3fa4307-9960-4456-9df9-6e8f5336de77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 23:02:09.565486 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fa4307-9960-4456-9df9-6e8f5336de77 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:02:09.569141 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564a952e-a4db-4cd2-94e9-9e426f29def4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:02:09.569779 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564a952e-a4db-4cd2-94e9-9e426f29def4 service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:02:09.570063 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564a952e-a4db-4cd2-94e9-9e426f29def4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:02:09.570341 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564a952e-a4db-4cd2-94e9-9e426f29def4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:02:09.579243 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564a952e-a4db-4cd2-94e9-9e426f29def4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:02:09.580808 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564a952e-a4db-4cd2-94e9-9e426f29def4 service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:02:09.590321 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1412/2828] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 23:02:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 23:02:10.031747 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b82d39-fb14-4590-a465-8a4e259b2fca None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:02:10.033225 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-b8b82d39-fb14-4590-a465-8a4e259b2fca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 23:02:10.047493 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b82d39-fb14-4590-a465-8a4e259b2fca None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:02:10.052199 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea7c0ce-fd5d-4525-93dc-b3f900fb00aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:02:10.053639 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea7c0ce-fd5d-4525-93dc-b3f900fb00aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:02:10.060518 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea7c0ce-fd5d-4525-93dc-b3f900fb00aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:02:10.060518 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea7c0ce-fd5d-4525-93dc-b3f900fb00aa admin admin] PATH_INFO: `/v3/projects/8a9b7dd3df944818b1f5315ff3e07011` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:02:10.060518 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea7c0ce-fd5d-4525-93dc-b3f900fb00aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a9b7dd3df944818b1f5315ff3e07011)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:02:10.060518 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea7c0ce-fd5d-4525-93dc-b3f900fb00aa admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:02:10.187533 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-1ea7c0ce-fd5d-4525-93dc-b3f900fb00aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a9b7dd3df944818b1f5315ff3e07011', 'request_id': 'req-1ea7c0ce-fd5d-4525-93dc-b3f900fb00aa'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 23:02:10.194297 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-1ea7c0ce-fd5d-4525-93dc-b3f900fb00aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a9b7dd3df944818b1f5315ff3e07011', 'request_id': 'req-1ea7c0ce-fd5d-4525-93dc-b3f900fb00aa'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 23:02:10.279020 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-1ea7c0ce-fd5d-4525-93dc-b3f900fb00aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a9b7dd3df944818b1f5315ff3e07011', 'request_id': 'req-1ea7c0ce-fd5d-4525-93dc-b3f900fb00aa'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 23:02:10.289989 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-1ea7c0ce-fd5d-4525-93dc-b3f900fb00aa admin admin] The token cache is being invalidate because project 8a9b7dd3df944818b1f5315ff3e07011 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 23:02:10.290274 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-1ea7c0ce-fd5d-4525-93dc-b3f900fb00aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 23:02:10.290926 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-1ea7c0ce-fd5d-4525-93dc-b3f900fb00aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 23:02:10.292482 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1417/2829] 10.222.0.81 () {58 vars in 1232 bytes} [Wed Jul 20 23:02:10 2022] DELETE /identity/v3/projects/8a9b7dd3df944818b1f5315ff3e07011 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 23:02:10.296466 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564a952e-a4db-4cd2-94e9-9e426f29def4 service neutron] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:02:10.306651 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564a952e-a4db-4cd2-94e9-9e426f29def4 service neutron] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:02:10.361601 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5192416-c8b7-427a-a809-441617265500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:02:10.362474 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5192416-c8b7-427a-a809-441617265500 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:02:10.362755 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5192416-c8b7-427a-a809-441617265500 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:02:10.363100 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5192416-c8b7-427a-a809-441617265500 admin admin] PATH_INFO: `/v3/users/32fcc71d27874a588ad61d617dd3cf2c` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:02:10.364279 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5192416-c8b7-427a-a809-441617265500 admin admin] RBAC: Authorizing `identity:delete_user(user_id=32fcc71d27874a588ad61d617dd3cf2c)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:02:10.366208 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5192416-c8b7-427a-a809-441617265500 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:02:10.589136 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-b5192416-c8b7-427a-a809-441617265500 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32fcc71d27874a588ad61d617dd3cf2c'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 23:02:10.589821 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-b5192416-c8b7-427a-a809-441617265500 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32fcc71d27874a588ad61d617dd3cf2c'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 23:02:10.590429 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-b5192416-c8b7-427a-a809-441617265500 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32fcc71d27874a588ad61d617dd3cf2c'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 23:02:10.598148 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-b5192416-c8b7-427a-a809-441617265500 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32fcc71d27874a588ad61d617dd3cf2c'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 23:02:10.684406 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-b5192416-c8b7-427a-a809-441617265500 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32fcc71d27874a588ad61d617dd3cf2c'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 23:02:10.689780 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 23:02:10.702110 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1413/2830] 10.222.0.81 () {58 vars in 1223 bytes} [Wed Jul 20 23:02:10 2022] DELETE /identity/v3/users/32fcc71d27874a588ad61d617dd3cf2c => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 23:02:11.195505 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea7c0ce-fd5d-4525-93dc-b3f900fb00aa admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:02:11.197199 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-1ea7c0ce-fd5d-4525-93dc-b3f900fb00aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 23:02:11.211975 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea7c0ce-fd5d-4525-93dc-b3f900fb00aa admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:02:11.224510 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 23:02:11.272212 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4a7f07-f1f9-4115-acbb-85793575ad50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:02:11.273101 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4a7f07-f1f9-4115-acbb-85793575ad50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:02:11.273382 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4a7f07-f1f9-4115-acbb-85793575ad50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:02:11.273658 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4a7f07-f1f9-4115-acbb-85793575ad50 admin admin] PATH_INFO: `/v3/projects/30872aaaa4ce440cb3ee682da6d03e6f` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:02:11.277259 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4a7f07-f1f9-4115-acbb-85793575ad50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=30872aaaa4ce440cb3ee682da6d03e6f)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:02:11.279358 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4a7f07-f1f9-4115-acbb-85793575ad50 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:02:11.422440 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-4c4a7f07-f1f9-4115-acbb-85793575ad50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30872aaaa4ce440cb3ee682da6d03e6f', 'request_id': 'req-4c4a7f07-f1f9-4115-acbb-85793575ad50'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 23:02:11.429326 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-4c4a7f07-f1f9-4115-acbb-85793575ad50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30872aaaa4ce440cb3ee682da6d03e6f', 'request_id': 'req-4c4a7f07-f1f9-4115-acbb-85793575ad50'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 23:02:11.542448 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-4c4a7f07-f1f9-4115-acbb-85793575ad50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30872aaaa4ce440cb3ee682da6d03e6f', 'request_id': 'req-4c4a7f07-f1f9-4115-acbb-85793575ad50'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 23:02:11.553542 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-4c4a7f07-f1f9-4115-acbb-85793575ad50 admin admin] The token cache is being invalidate because project 30872aaaa4ce440cb3ee682da6d03e6f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 23:02:11.563301 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-4c4a7f07-f1f9-4115-acbb-85793575ad50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 23:02:11.563301 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-4c4a7f07-f1f9-4115-acbb-85793575ad50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 23:02:11.563301 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1418/2831] 10.222.0.81 () {58 vars in 1232 bytes} [Wed Jul 20 23:02:11 2022] DELETE /identity/v3/projects/30872aaaa4ce440cb3ee682da6d03e6f => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 23:02:11.563683 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296c9470-9567-4e57-8ec2-63ab3b61b06c None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:02:11.563683 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296c9470-9567-4e57-8ec2-63ab3b61b06c None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:02:11.563683 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296c9470-9567-4e57-8ec2-63ab3b61b06c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:02:11.579506 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-296c9470-9567-4e57-8ec2-63ab3b61b06c None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 23:02:11.580747 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-296c9470-9567-4e57-8ec2-63ab3b61b06c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 23:02:11.644084 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1414/2832] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 23:02:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 23:02:11.652076 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a7dd7e-f02e-4c48-b240-7ab1958d6c4b None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:02:11.652379 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a7dd7e-f02e-4c48-b240-7ab1958d6c4b None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:02:11.652663 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a7dd7e-f02e-4c48-b240-7ab1958d6c4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:02:11.671195 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-a3a7dd7e-f02e-4c48-b240-7ab1958d6c4b None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 23:02:11.672445 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-a3a7dd7e-f02e-4c48-b240-7ab1958d6c4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 23:02:11.681053 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1419/2833] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 23:02:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 23:02:11.685286 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296c9470-9567-4e57-8ec2-63ab3b61b06c None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:02:11.694857 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296c9470-9567-4e57-8ec2-63ab3b61b06c None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:02:11.699436 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f752d237-ee25-4fb2-98b4-52cb2a089635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:02:11.700952 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f752d237-ee25-4fb2-98b4-52cb2a089635 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:02:11.700952 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f752d237-ee25-4fb2-98b4-52cb2a089635 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:02:11.700952 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f752d237-ee25-4fb2-98b4-52cb2a089635 admin admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:02:11.701405 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f752d237-ee25-4fb2-98b4-52cb2a089635 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 23:02:11.701716 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f752d237-ee25-4fb2-98b4-52cb2a089635 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:02:11.703339 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f752d237-ee25-4fb2-98b4-52cb2a089635 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:02:11.715382 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1415/2834] 10.222.0.81 () {58 vars in 1152 bytes} [Wed Jul 20 23:02:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 23:02:11.720691 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a7dd7e-f02e-4c48-b240-7ab1958d6c4b None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:02:11.729604 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a7dd7e-f02e-4c48-b240-7ab1958d6c4b None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:02:11.734343 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87f4c3e-7e8b-4854-8a5e-313708d1b604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:02:11.735000 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87f4c3e-7e8b-4854-8a5e-313708d1b604 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:02:11.735316 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87f4c3e-7e8b-4854-8a5e-313708d1b604 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:02:11.735609 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87f4c3e-7e8b-4854-8a5e-313708d1b604 admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:02:11.736379 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87f4c3e-7e8b-4854-8a5e-313708d1b604 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:02:11.738262 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87f4c3e-7e8b-4854-8a5e-313708d1b604 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:02:11.823109 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1420/2835] 10.222.0.81 () {60 vars in 1152 bytes} [Wed Jul 20 23:02:11 2022] POST /identity/v3/projects => generated 361 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 23:02:11.827461 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f752d237-ee25-4fb2-98b4-52cb2a089635 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:02:11.838006 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f752d237-ee25-4fb2-98b4-52cb2a089635 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:02:11.893291 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e83b334-c24e-42f5-8a4e-5e105de25024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:02:11.893950 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e83b334-c24e-42f5-8a4e-5e105de25024 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:02:11.894235 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e83b334-c24e-42f5-8a4e-5e105de25024 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:02:11.894516 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e83b334-c24e-42f5-8a4e-5e105de25024 admin admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:02:11.895233 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e83b334-c24e-42f5-8a4e-5e105de25024 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:02:11.897144 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e83b334-c24e-42f5-8a4e-5e105de25024 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:02:11.987323 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1416/2836] 10.222.0.81 () {60 vars in 1143 bytes} [Wed Jul 20 23:02:11 2022] POST /identity/v3/users => generated 281 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 23:02:11.991437 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87f4c3e-7e8b-4854-8a5e-313708d1b604 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:02:12.001569 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87f4c3e-7e8b-4854-8a5e-313708d1b604 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:02:12.006313 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cc6ab4-f471-4060-9b89-4a4c933e6ee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:02:12.006958 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cc6ab4-f471-4060-9b89-4a4c933e6ee5 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:02:12.007274 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cc6ab4-f471-4060-9b89-4a4c933e6ee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:02:12.007553 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cc6ab4-f471-4060-9b89-4a4c933e6ee5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:02:12.008630 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cc6ab4-f471-4060-9b89-4a4c933e6ee5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:02:12.010359 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cc6ab4-f471-4060-9b89-4a4c933e6ee5 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:02:12.021353 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1421/2837] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 23:02:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 23:02:12.025182 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e83b334-c24e-42f5-8a4e-5e105de25024 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:02:12.036039 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e83b334-c24e-42f5-8a4e-5e105de25024 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:02:12.040714 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b70efb-0703-48f9-96dd-4f5be4822a0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:02:12.041376 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b70efb-0703-48f9-96dd-4f5be4822a0c admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:02:12.041666 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b70efb-0703-48f9-96dd-4f5be4822a0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:02:12.041946 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b70efb-0703-48f9-96dd-4f5be4822a0c admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:02:12.042670 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b70efb-0703-48f9-96dd-4f5be4822a0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:02:12.044425 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b70efb-0703-48f9-96dd-4f5be4822a0c admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:02:12.054444 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1417/2838] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 23:02:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 23:02:12.058465 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cc6ab4-f471-4060-9b89-4a4c933e6ee5 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:02:12.068447 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cc6ab4-f471-4060-9b89-4a4c933e6ee5 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:02:12.073103 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af05bf3b-5f7e-4c63-9559-8f358df69a6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:02:12.074034 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af05bf3b-5f7e-4c63-9559-8f358df69a6c admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:02:12.074273 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af05bf3b-5f7e-4c63-9559-8f358df69a6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:02:12.074576 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af05bf3b-5f7e-4c63-9559-8f358df69a6c admin admin] PATH_INFO: `/v3/projects/51b18d1b79c249938bd5641114ba737f/users/95c7097d30cd457fbf74fcb112cadec5/roles/559942c4de1744169f8b5e0404ecde6b` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:02:12.090186 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af05bf3b-5f7e-4c63-9559-8f358df69a6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=51b18d1b79c249938bd5641114ba737f, user_id=95c7097d30cd457fbf74fcb112cadec5, role_id=559942c4de1744169f8b5e0404ecde6b)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:02:12.092838 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af05bf3b-5f7e-4c63-9559-8f358df69a6c admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:02:12.177077 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1422/2839] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 23:02:12 2022] PUT /identity/v3/projects/51b18d1b79c249938bd5641114ba737f/users/95c7097d30cd457fbf74fcb112cadec5/roles/559942c4de1744169f8b5e0404ecde6b => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 23:02:12.238527 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f04c422-88a1-4563-8c62-ff95b5b6cb06 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:02:12.238824 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f04c422-88a1-4563-8c62-ff95b5b6cb06 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:02:12.239140 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f04c422-88a1-4563-8c62-ff95b5b6cb06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:02:12.266044 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-5f04c422-88a1-4563-8c62-ff95b5b6cb06 None None] MFA Rules not processed for user `95c7097d30cd457fbf74fcb112cadec5`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 23:02:12.267883 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-5f04c422-88a1-4563-8c62-ff95b5b6cb06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 23:02:12.311245 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 23:02:12.334863 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1418/2840] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 23:02:12 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 23:02:12.365818 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af05bf3b-5f7e-4c63-9559-8f358df69a6c admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:02:12.367947 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-af05bf3b-5f7e-4c63-9559-8f358df69a6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 23:02:12.415449 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 23:02:12.427209 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af05bf3b-5f7e-4c63-9559-8f358df69a6c admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:02:12.430745 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb12d1a2-7480-4f34-89cd-e00d70ced8c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecf4db289fba4e4a806198c62d31f7c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:02:12.433545 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb12d1a2-7480-4f34-89cd-e00d70ced8c5 service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:02:12.433545 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb12d1a2-7480-4f34-89cd-e00d70ced8c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:02:12.433545 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb12d1a2-7480-4f34-89cd-e00d70ced8c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:02:12.440765 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb12d1a2-7480-4f34-89cd-e00d70ced8c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:02:12.442214 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb12d1a2-7480-4f34-89cd-e00d70ced8c5 service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:02:12.451712 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1423/2841] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 23:02:12 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 23:02:12.466893 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f04c422-88a1-4563-8c62-ff95b5b6cb06 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:02:12.529324 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f04c422-88a1-4563-8c62-ff95b5b6cb06 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:02:12.534033 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5474fb48-f5db-4b7d-8712-e4d09e76d588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:02:12.534914 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5474fb48-f5db-4b7d-8712-e4d09e76d588 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:02:12.535230 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5474fb48-f5db-4b7d-8712-e4d09e76d588 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:02:12.535512 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5474fb48-f5db-4b7d-8712-e4d09e76d588 admin admin] PATH_INFO: `/v3/users/95c7097d30cd457fbf74fcb112cadec5` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:02:12.536690 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5474fb48-f5db-4b7d-8712-e4d09e76d588 admin admin] RBAC: Authorizing `identity:delete_user(user_id=95c7097d30cd457fbf74fcb112cadec5)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:02:12.538627 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5474fb48-f5db-4b7d-8712-e4d09e76d588 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:02:12.737879 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-5474fb48-f5db-4b7d-8712-e4d09e76d588 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95c7097d30cd457fbf74fcb112cadec5'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 23:02:12.738553 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-5474fb48-f5db-4b7d-8712-e4d09e76d588 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95c7097d30cd457fbf74fcb112cadec5'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 23:02:12.739085 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-5474fb48-f5db-4b7d-8712-e4d09e76d588 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95c7097d30cd457fbf74fcb112cadec5'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 23:02:12.745971 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-5474fb48-f5db-4b7d-8712-e4d09e76d588 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95c7097d30cd457fbf74fcb112cadec5'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 23:02:12.821313 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-5474fb48-f5db-4b7d-8712-e4d09e76d588 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95c7097d30cd457fbf74fcb112cadec5'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 23:02:12.838743 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1419/2842] 10.222.0.81 () {58 vars in 1223 bytes} [Wed Jul 20 23:02:12 2022] DELETE /identity/v3/users/95c7097d30cd457fbf74fcb112cadec5 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 23:02:12.843274 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ca6514-173b-474d-813e-5367b3232677 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:02:12.843550 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ca6514-173b-474d-813e-5367b3232677 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:02:12.843830 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ca6514-173b-474d-813e-5367b3232677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:02:12.861356 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-24ca6514-173b-474d-813e-5367b3232677 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 23:02:12.862562 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-24ca6514-173b-474d-813e-5367b3232677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 23:02:12.919093 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1424/2843] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 23:02:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 23:02:12.933019 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5474fb48-f5db-4b7d-8712-e4d09e76d588 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:02:12.934544 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-5474fb48-f5db-4b7d-8712-e4d09e76d588 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 23:02:12.991700 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5474fb48-f5db-4b7d-8712-e4d09e76d588 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:02:12.995196 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3784d1e-bd5e-44de-a41b-7bd10f334bd9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:02:12.995840 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3784d1e-bd5e-44de-a41b-7bd10f334bd9 service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:02:12.996122 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3784d1e-bd5e-44de-a41b-7bd10f334bd9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:02:12.996401 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3784d1e-bd5e-44de-a41b-7bd10f334bd9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:02:13.005822 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3784d1e-bd5e-44de-a41b-7bd10f334bd9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:02:13.007261 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3784d1e-bd5e-44de-a41b-7bd10f334bd9 service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:02:13.016802 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1420/2844] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 23:02:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 23:02:13.440956 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ca6514-173b-474d-813e-5367b3232677 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:02:13.442539 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-24ca6514-173b-474d-813e-5367b3232677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 23:02:13.455109 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ca6514-173b-474d-813e-5367b3232677 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:02:13.459535 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094c6f4d-d58e-4454-89b5-c9816ce224f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:02:13.460416 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094c6f4d-d58e-4454-89b5-c9816ce224f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:02:13.460885 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094c6f4d-d58e-4454-89b5-c9816ce224f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:02:13.461166 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094c6f4d-d58e-4454-89b5-c9816ce224f4 admin admin] PATH_INFO: `/v3/projects/51b18d1b79c249938bd5641114ba737f` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:02:13.462455 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094c6f4d-d58e-4454-89b5-c9816ce224f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=51b18d1b79c249938bd5641114ba737f)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:02:13.464389 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094c6f4d-d58e-4454-89b5-c9816ce224f4 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:02:13.593081 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-094c6f4d-d58e-4454-89b5-c9816ce224f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51b18d1b79c249938bd5641114ba737f', 'request_id': 'req-094c6f4d-d58e-4454-89b5-c9816ce224f4'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 23:02:13.599947 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-094c6f4d-d58e-4454-89b5-c9816ce224f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51b18d1b79c249938bd5641114ba737f', 'request_id': 'req-094c6f4d-d58e-4454-89b5-c9816ce224f4'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 23:02:13.679058 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-094c6f4d-d58e-4454-89b5-c9816ce224f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51b18d1b79c249938bd5641114ba737f', 'request_id': 'req-094c6f4d-d58e-4454-89b5-c9816ce224f4'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 23:02:13.689672 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-094c6f4d-d58e-4454-89b5-c9816ce224f4 admin admin] The token cache is being invalidate because project 51b18d1b79c249938bd5641114ba737f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 23:02:13.694220 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-094c6f4d-d58e-4454-89b5-c9816ce224f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 23:02:13.694220 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-094c6f4d-d58e-4454-89b5-c9816ce224f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 23:02:13.694220 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1425/2845] 10.222.0.81 () {58 vars in 1232 bytes} [Wed Jul 20 23:02:13 2022] DELETE /identity/v3/projects/51b18d1b79c249938bd5641114ba737f => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 23:02:13.697195 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d8d330-11e7-4b88-aaa6-16b3f589fa68 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:02:13.697515 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d8d330-11e7-4b88-aaa6-16b3f589fa68 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:02:13.697800 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d8d330-11e7-4b88-aaa6-16b3f589fa68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:02:13.705070 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 23:02:13.715851 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-03d8d330-11e7-4b88-aaa6-16b3f589fa68 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 23:02:13.717042 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-03d8d330-11e7-4b88-aaa6-16b3f589fa68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 23:02:13.774690 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1421/2846] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 23:02:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 23:02:13.783093 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7ca5b0-4493-4a2c-867e-a9940824efc0 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:02:13.783387 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7ca5b0-4493-4a2c-867e-a9940824efc0 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:02:13.783673 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7ca5b0-4493-4a2c-867e-a9940824efc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:02:13.802466 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-1c7ca5b0-4493-4a2c-867e-a9940824efc0 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 23:02:13.803725 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-1c7ca5b0-4493-4a2c-867e-a9940824efc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 23:02:13.811708 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1426/2847] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 23:02:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 23:02:13.816705 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d8d330-11e7-4b88-aaa6-16b3f589fa68 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:02:13.826361 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d8d330-11e7-4b88-aaa6-16b3f589fa68 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:02:13.831586 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463f713b-549b-4ed9-937a-9114417dd536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:02:13.832312 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463f713b-549b-4ed9-937a-9114417dd536 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:02:13.832801 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463f713b-549b-4ed9-937a-9114417dd536 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:02:13.833328 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463f713b-549b-4ed9-937a-9114417dd536 admin admin] PATH_INFO: `/v3/domains` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:02:13.834440 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463f713b-549b-4ed9-937a-9114417dd536 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80441) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 23:02:13.834808 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463f713b-549b-4ed9-937a-9114417dd536 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:02:13.836439 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463f713b-549b-4ed9-937a-9114417dd536 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:02:13.847682 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1422/2848] 10.222.0.81 () {58 vars in 1152 bytes} [Wed Jul 20 23:02:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 23:02:13.852552 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7ca5b0-4493-4a2c-867e-a9940824efc0 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:02:13.861915 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7ca5b0-4493-4a2c-867e-a9940824efc0 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:02:13.866548 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebb2495-25fa-4d5d-81d5-fa422ba19009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:02:13.867252 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebb2495-25fa-4d5d-81d5-fa422ba19009 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:02:13.867541 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebb2495-25fa-4d5d-81d5-fa422ba19009 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:02:13.867814 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebb2495-25fa-4d5d-81d5-fa422ba19009 admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:02:13.868501 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebb2495-25fa-4d5d-81d5-fa422ba19009 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:02:13.870376 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebb2495-25fa-4d5d-81d5-fa422ba19009 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:02:13.988994 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1427/2849] 10.222.0.81 () {60 vars in 1152 bytes} [Wed Jul 20 23:02:13 2022] POST /identity/v3/projects => generated 383 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 23:02:13.993139 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463f713b-549b-4ed9-937a-9114417dd536 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:02:14.003985 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463f713b-549b-4ed9-937a-9114417dd536 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:02:14.062542 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7e7bcd-4ba3-40cb-9da6-9c20a69e9bbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:02:14.063239 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7e7bcd-4ba3-40cb-9da6-9c20a69e9bbb admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:02:14.063524 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7e7bcd-4ba3-40cb-9da6-9c20a69e9bbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:02:14.064350 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7e7bcd-4ba3-40cb-9da6-9c20a69e9bbb admin admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:02:14.065042 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7e7bcd-4ba3-40cb-9da6-9c20a69e9bbb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:02:14.066942 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7e7bcd-4ba3-40cb-9da6-9c20a69e9bbb admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:02:14.188338 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1423/2850] 10.222.0.81 () {60 vars in 1143 bytes} [Wed Jul 20 23:02:13 2022] POST /identity/v3/users => generated 292 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 23:02:14.192511 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebb2495-25fa-4d5d-81d5-fa422ba19009 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:02:14.203031 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebb2495-25fa-4d5d-81d5-fa422ba19009 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:02:14.208291 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c67fdff-fb1f-4ecd-bccd-94f50e62a0ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:02:14.208963 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c67fdff-fb1f-4ecd-bccd-94f50e62a0ac admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:02:14.209250 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c67fdff-fb1f-4ecd-bccd-94f50e62a0ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:02:14.209531 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c67fdff-fb1f-4ecd-bccd-94f50e62a0ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:02:14.210256 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c67fdff-fb1f-4ecd-bccd-94f50e62a0ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:02:14.211880 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c67fdff-fb1f-4ecd-bccd-94f50e62a0ac admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:02:14.222890 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1428/2851] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 23:02:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 23:02:14.227301 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7e7bcd-4ba3-40cb-9da6-9c20a69e9bbb admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:02:14.237395 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7e7bcd-4ba3-40cb-9da6-9c20a69e9bbb admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:02:14.242223 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3467906-f486-47a9-a089-fc32c80ad678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:02:14.242920 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3467906-f486-47a9-a089-fc32c80ad678 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:02:14.243259 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3467906-f486-47a9-a089-fc32c80ad678 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:02:14.243543 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3467906-f486-47a9-a089-fc32c80ad678 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:02:14.244280 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3467906-f486-47a9-a089-fc32c80ad678 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:02:14.245904 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3467906-f486-47a9-a089-fc32c80ad678 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:02:14.255830 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1424/2852] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 23:02:14 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 23:02:14.260029 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c67fdff-fb1f-4ecd-bccd-94f50e62a0ac admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:02:14.270316 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c67fdff-fb1f-4ecd-bccd-94f50e62a0ac admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:02:14.274882 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5442ee29-1a73-43d8-8547-6836ac6169b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:02:14.275803 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5442ee29-1a73-43d8-8547-6836ac6169b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:02:14.276089 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5442ee29-1a73-43d8-8547-6836ac6169b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:02:14.276365 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5442ee29-1a73-43d8-8547-6836ac6169b8 admin admin] PATH_INFO: `/v3/projects/62f889bb65b048cb96e5548e4420df8f/users/07feb13dc066470cae9d68d0d38d89cb/roles/559942c4de1744169f8b5e0404ecde6b` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:02:14.291385 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5442ee29-1a73-43d8-8547-6836ac6169b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62f889bb65b048cb96e5548e4420df8f, user_id=07feb13dc066470cae9d68d0d38d89cb, role_id=559942c4de1744169f8b5e0404ecde6b)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:02:14.294033 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5442ee29-1a73-43d8-8547-6836ac6169b8 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:02:14.382454 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1429/2853] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 23:02:14 2022] PUT /identity/v3/projects/62f889bb65b048cb96e5548e4420df8f/users/07feb13dc066470cae9d68d0d38d89cb/roles/559942c4de1744169f8b5e0404ecde6b => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 23:02:14.387795 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de19647b-064d-41ea-8066-9b2debec3c97 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:02:14.388092 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de19647b-064d-41ea-8066-9b2debec3c97 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:02:14.388377 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de19647b-064d-41ea-8066-9b2debec3c97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:02:14.406949 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-de19647b-064d-41ea-8066-9b2debec3c97 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 23:02:14.408239 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-de19647b-064d-41ea-8066-9b2debec3c97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 23:02:14.466465 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1425/2854] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 23:02:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 23:02:14.476102 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5442ee29-1a73-43d8-8547-6836ac6169b8 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:02:14.477850 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-5442ee29-1a73-43d8-8547-6836ac6169b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 23:02:14.484382 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 23:02:14.533270 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5442ee29-1a73-43d8-8547-6836ac6169b8 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:02:14.536954 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4a6086-5a80-4075-8f3a-b0a14366268d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:02:14.537603 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4a6086-5a80-4075-8f3a-b0a14366268d service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:02:14.537893 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4a6086-5a80-4075-8f3a-b0a14366268d service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:02:14.538179 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4a6086-5a80-4075-8f3a-b0a14366268d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:02:14.546989 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4a6086-5a80-4075-8f3a-b0a14366268d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:02:14.548434 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4a6086-5a80-4075-8f3a-b0a14366268d service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:02:14.557832 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1430/2855] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 23:02:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 23:02:20.526654 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c5346e-4ce4-4ce2-ae75-d9fa95cfd7b1 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:02:20.527175 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c5346e-4ce4-4ce2-ae75-d9fa95cfd7b1 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:02:20.527301 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c5346e-4ce4-4ce2-ae75-d9fa95cfd7b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:02:20.547466 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 23:02:20.556156 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-43c5346e-4ce4-4ce2-ae75-d9fa95cfd7b1 None None] MFA Rules not processed for user `07feb13dc066470cae9d68d0d38d89cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 23:02:20.557384 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-43c5346e-4ce4-4ce2-ae75-d9fa95cfd7b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 23:02:20.628555 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1426/2856] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 23:02:20 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 23:02:20.630104 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4a6086-5a80-4075-8f3a-b0a14366268d service neutron] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:02:20.641280 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4a6086-5a80-4075-8f3a-b0a14366268d service neutron] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:02:20.646334 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4f2378-a991-40d2-a6f4-fe3f9d94c106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:02:20.647158 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4f2378-a991-40d2-a6f4-fe3f9d94c106 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:02:20.647444 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4f2378-a991-40d2-a6f4-fe3f9d94c106 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:02:20.647736 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4f2378-a991-40d2-a6f4-fe3f9d94c106 admin admin] PATH_INFO: `/v3/projects` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:02:20.648422 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4f2378-a991-40d2-a6f4-fe3f9d94c106 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:02:20.650449 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4f2378-a991-40d2-a6f4-fe3f9d94c106 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:02:20.754909 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1431/2857] 10.222.0.81 () {60 vars in 1152 bytes} [Wed Jul 20 23:02:20 2022] POST /identity/v3/projects => generated 385 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 23:02:20.759166 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c5346e-4ce4-4ce2-ae75-d9fa95cfd7b1 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:02:20.770496 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c5346e-4ce4-4ce2-ae75-d9fa95cfd7b1 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:02:20.827557 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a55a37-fec4-492f-ad5a-a62332fac4b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:02:20.828251 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a55a37-fec4-492f-ad5a-a62332fac4b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:02:20.828539 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a55a37-fec4-492f-ad5a-a62332fac4b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:02:20.828820 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a55a37-fec4-492f-ad5a-a62332fac4b6 admin admin] PATH_INFO: `/v3/users` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:02:20.830029 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a55a37-fec4-492f-ad5a-a62332fac4b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:02:20.831997 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a55a37-fec4-492f-ad5a-a62332fac4b6 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:02:20.948211 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1427/2858] 10.222.0.81 () {60 vars in 1143 bytes} [Wed Jul 20 23:02:20 2022] POST /identity/v3/users => generated 299 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 23:02:20.952198 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4f2378-a991-40d2-a6f4-fe3f9d94c106 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:02:20.963476 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4f2378-a991-40d2-a6f4-fe3f9d94c106 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:02:20.968154 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ec05be-9465-4239-9d7d-ffc2007f2f4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:02:20.968963 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ec05be-9465-4239-9d7d-ffc2007f2f4d admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:02:20.969265 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ec05be-9465-4239-9d7d-ffc2007f2f4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:02:20.969537 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ec05be-9465-4239-9d7d-ffc2007f2f4d admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:02:20.970881 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ec05be-9465-4239-9d7d-ffc2007f2f4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:02:20.972534 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ec05be-9465-4239-9d7d-ffc2007f2f4d admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:02:20.984255 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1432/2859] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 23:02:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 23:02:20.988353 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a55a37-fec4-492f-ad5a-a62332fac4b6 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:02:20.998709 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a55a37-fec4-492f-ad5a-a62332fac4b6 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:02:21.003725 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8527c501-5842-445f-9a65-820e6efba72a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:02:21.004623 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8527c501-5842-445f-9a65-820e6efba72a admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:02:21.004961 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8527c501-5842-445f-9a65-820e6efba72a admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:02:21.005191 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8527c501-5842-445f-9a65-820e6efba72a admin admin] PATH_INFO: `/v3/projects/59dfcc8c99e2485aab28879af32e2275/users/6c4e16396f3f41dfacf0407220948e27/roles/d89b6c92d35a401a8517f1d47f6be843` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:02:21.020600 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8527c501-5842-445f-9a65-820e6efba72a admin admin] RBAC: Authorizing `identity:create_grant(project_id=59dfcc8c99e2485aab28879af32e2275, user_id=6c4e16396f3f41dfacf0407220948e27, role_id=d89b6c92d35a401a8517f1d47f6be843)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:02:21.023267 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8527c501-5842-445f-9a65-820e6efba72a admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:02:21.097046 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1428/2860] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 23:02:20 2022] PUT /identity/v3/projects/59dfcc8c99e2485aab28879af32e2275/users/6c4e16396f3f41dfacf0407220948e27/roles/d89b6c92d35a401a8517f1d47f6be843 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 23:02:21.101040 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ec05be-9465-4239-9d7d-ffc2007f2f4d admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:02:21.112457 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ec05be-9465-4239-9d7d-ffc2007f2f4d admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:02:21.172032 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068003fa-05df-4e96-82b4-3681a5887026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:02:21.172712 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068003fa-05df-4e96-82b4-3681a5887026 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:02:21.173583 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068003fa-05df-4e96-82b4-3681a5887026 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:02:21.173865 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068003fa-05df-4e96-82b4-3681a5887026 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:02:21.174681 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068003fa-05df-4e96-82b4-3681a5887026 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:02:21.176313 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068003fa-05df-4e96-82b4-3681a5887026 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:02:21.186817 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1433/2861] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 23:02:21 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 23:02:21.192910 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8527c501-5842-445f-9a65-820e6efba72a admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:02:21.203636 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8527c501-5842-445f-9a65-820e6efba72a admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:02:21.208610 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c021dad-3a54-41b3-a550-7079c48e9470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:02:21.209621 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c021dad-3a54-41b3-a550-7079c48e9470 admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:02:21.209930 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c021dad-3a54-41b3-a550-7079c48e9470 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:02:21.210194 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c021dad-3a54-41b3-a550-7079c48e9470 admin admin] PATH_INFO: `/v3/projects/59dfcc8c99e2485aab28879af32e2275/users/6c4e16396f3f41dfacf0407220948e27/roles/d89b6c92d35a401a8517f1d47f6be843` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:02:21.212223 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c021dad-3a54-41b3-a550-7079c48e9470 admin admin] RBAC: Authorizing `identity:create_grant(project_id=59dfcc8c99e2485aab28879af32e2275, user_id=6c4e16396f3f41dfacf0407220948e27, role_id=d89b6c92d35a401a8517f1d47f6be843)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:02:21.214920 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c021dad-3a54-41b3-a550-7079c48e9470 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:02:21.226327 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1429/2862] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 23:02:21 2022] PUT /identity/v3/projects/59dfcc8c99e2485aab28879af32e2275/users/6c4e16396f3f41dfacf0407220948e27/roles/d89b6c92d35a401a8517f1d47f6be843 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 23:02:27.124661 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de07309-05f3-4827-94e3-a80817a67f76 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:02:27.125257 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de07309-05f3-4827-94e3-a80817a67f76 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:02:27.125369 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de07309-05f3-4827-94e3-a80817a67f76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:02:27.156407 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-9de07309-05f3-4827-94e3-a80817a67f76 None None] MFA Rules not processed for user `6c4e16396f3f41dfacf0407220948e27`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 23:02:27.159242 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-9de07309-05f3-4827-94e3-a80817a67f76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 23:02:27.233246 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 23:02:27.242226 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1434/2863] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 23:02:27 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 23:02:27.255809 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c021dad-3a54-41b3-a550-7079c48e9470 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:02:27.257740 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-8c021dad-3a54-41b3-a550-7079c48e9470 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 23:02:27.319695 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 23:02:27.342447 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c021dad-3a54-41b3-a550-7079c48e9470 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:02:27.347835 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8feaad2f-d66b-4810-8514-09b3deefa5de service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9c44640ea0014426a9523bb1c457778b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:02:27.347835 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feaad2f-d66b-4810-8514-09b3deefa5de service glance] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:02:27.347835 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feaad2f-d66b-4810-8514-09b3deefa5de service glance] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:02:27.348369 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feaad2f-d66b-4810-8514-09b3deefa5de service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:02:27.405572 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8feaad2f-d66b-4810-8514-09b3deefa5de service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:02:27.407106 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8feaad2f-d66b-4810-8514-09b3deefa5de service glance] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:02:27.416296 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1430/2864] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 23:02:27 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 23:02:27.679712 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6f7e9b-a1e6-46f8-9ea5-21140ff7022b None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:02:27.680008 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6f7e9b-a1e6-46f8-9ea5-21140ff7022b None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:02:27.680290 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6f7e9b-a1e6-46f8-9ea5-21140ff7022b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:02:27.699481 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-ce6f7e9b-a1e6-46f8-9ea5-21140ff7022b None None] MFA Rules not processed for user `90915e1264324f16bc7bae7e46e0c97b`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 23:02:27.700692 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-ce6f7e9b-a1e6-46f8-9ea5-21140ff7022b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 23:02:27.755741 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1435/2865] 10.222.0.81 () {58 vars in 1009 bytes} [Wed Jul 20 23:02:27 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 23:02:27.968941 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8feaad2f-d66b-4810-8514-09b3deefa5de service glance] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:02:27.970961 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-8feaad2f-d66b-4810-8514-09b3deefa5de service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 23:02:28.043040 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8feaad2f-d66b-4810-8514-09b3deefa5de service glance] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:02:28.050556 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb66ce7-0a60-4d9c-abf7-fcda2f334e6e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70e97436a08543a180225561ef8bc817', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:02:28.050556 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb66ce7-0a60-4d9c-abf7-fcda2f334e6e service swift] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:02:28.050556 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb66ce7-0a60-4d9c-abf7-fcda2f334e6e service swift] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:02:28.050556 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb66ce7-0a60-4d9c-abf7-fcda2f334e6e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:02:28.058687 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb66ce7-0a60-4d9c-abf7-fcda2f334e6e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:02:28.060341 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb66ce7-0a60-4d9c-abf7-fcda2f334e6e service swift] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:02:28.070099 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1431/2866] 10.222.0.81 () {60 vars in 1384 bytes} [Wed Jul 20 23:02:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 23:02:37.748173 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=27 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 23:02:38.065433 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=65 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 23:02:38.454157 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f9c2b1-1e11-48cc-a3b2-7ac1c76a32b9 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:02:38.454485 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f9c2b1-1e11-48cc-a3b2-7ac1c76a32b9 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:02:38.454790 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f9c2b1-1e11-48cc-a3b2-7ac1c76a32b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:02:38.472965 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-12f9c2b1-1e11-48cc-a3b2-7ac1c76a32b9 None None] MFA Rules not processed for user `07feb13dc066470cae9d68d0d38d89cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 23:02:38.474181 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-12f9c2b1-1e11-48cc-a3b2-7ac1c76a32b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 23:02:38.481235 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1436/2867] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 23:02:38 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 23:02:39.174841 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb66ce7-0a60-4d9c-abf7-fcda2f334e6e service swift] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:02:39.176616 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-ecb66ce7-0a60-4d9c-abf7-fcda2f334e6e service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 23:02:39.251634 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb66ce7-0a60-4d9c-abf7-fcda2f334e6e service swift] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:02:39.256637 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e6e284-f4f0-4d2e-83ff-aca981f19ff1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5500cebdd9d4f8e9aecb45d6c955878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:02:39.262475 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e6e284-f4f0-4d2e-83ff-aca981f19ff1 service placement] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:02:39.262475 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e6e284-f4f0-4d2e-83ff-aca981f19ff1 service placement] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:02:39.262475 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e6e284-f4f0-4d2e-83ff-aca981f19ff1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:02:39.262475 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-88e6e284-f4f0-4d2e-83ff-aca981f19ff1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 23:02:39.271156 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e6e284-f4f0-4d2e-83ff-aca981f19ff1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:02:39.272724 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e6e284-f4f0-4d2e-83ff-aca981f19ff1 service placement] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:02:39.283075 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1432/2868] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 23:02:39 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 23:02:42.484617 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ada9945-e89a-4d28-bb7f-adcb7297d04c None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:02:42.485553 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ada9945-e89a-4d28-bb7f-adcb7297d04c None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:02:42.485903 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ada9945-e89a-4d28-bb7f-adcb7297d04c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:02:42.514023 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-4ada9945-e89a-4d28-bb7f-adcb7297d04c None None] MFA Rules not processed for user `90915e1264324f16bc7bae7e46e0c97b`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 23:02:42.515441 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-4ada9945-e89a-4d28-bb7f-adcb7297d04c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 23:02:42.523882 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1437/2869] 10.222.0.81 () {58 vars in 1009 bytes} [Wed Jul 20 23:02:42 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 23:02:42.537466 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e6e284-f4f0-4d2e-83ff-aca981f19ff1 service placement] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:02:42.550510 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e6e284-f4f0-4d2e-83ff-aca981f19ff1 service placement] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:02:42.556257 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04497c4a-d322-461d-8137-27918d64aeb2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70e97436a08543a180225561ef8bc817', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:02:42.556921 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04497c4a-d322-461d-8137-27918d64aeb2 service swift] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:02:42.557214 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04497c4a-d322-461d-8137-27918d64aeb2 service swift] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:02:42.557495 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04497c4a-d322-461d-8137-27918d64aeb2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:02:42.567149 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04497c4a-d322-461d-8137-27918d64aeb2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:02:42.568777 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04497c4a-d322-461d-8137-27918d64aeb2 service swift] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:02:42.578149 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1433/2870] 10.222.0.81 () {60 vars in 1384 bytes} [Wed Jul 20 23:02:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 23:02:42.743302 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ada9945-e89a-4d28-bb7f-adcb7297d04c None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:02:42.824568 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ada9945-e89a-4d28-bb7f-adcb7297d04c None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:02:42.829183 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7801d393-f04e-4955-a710-64d31d9b4106 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:02:42.830235 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7801d393-f04e-4955-a710-64d31d9b4106 service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:02:42.830620 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7801d393-f04e-4955-a710-64d31d9b4106 service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:02:42.830904 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7801d393-f04e-4955-a710-64d31d9b4106 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:02:42.834112 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-7801d393-f04e-4955-a710-64d31d9b4106 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 23:02:42.846925 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7801d393-f04e-4955-a710-64d31d9b4106 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:02:42.848455 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7801d393-f04e-4955-a710-64d31d9b4106 service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:02:42.862386 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1438/2871] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 23:02:42 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 23:02:43.829877 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04497c4a-d322-461d-8137-27918d64aeb2 service swift] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:02:43.831590 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-04497c4a-d322-461d-8137-27918d64aeb2 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 23:02:43.958349 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04497c4a-d322-461d-8137-27918d64aeb2 service swift] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:02:43.964897 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84949330-d6c7-451a-97e7-797ce47da559 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7829324db2574d38b18c749d0559177f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:02:43.965714 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84949330-d6c7-451a-97e7-797ce47da559 service nova] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:02:43.966005 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84949330-d6c7-451a-97e7-797ce47da559 service nova] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:02:43.966287 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84949330-d6c7-451a-97e7-797ce47da559 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:02:43.972499 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-84949330-d6c7-451a-97e7-797ce47da559 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 23:02:43.987908 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84949330-d6c7-451a-97e7-797ce47da559 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:02:43.989528 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84949330-d6c7-451a-97e7-797ce47da559 service nova] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:02:44.013363 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1434/2872] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 23:02:43 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 23:02:52.853900 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=32 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 23:02:53.997193 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=72 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 23:03:16.819950 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7801d393-f04e-4955-a710-64d31d9b4106 service neutron] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:03:16.821364 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-7801d393-f04e-4955-a710-64d31d9b4106 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 23:03:16.831883 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7801d393-f04e-4955-a710-64d31d9b4106 service neutron] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:03:16.835510 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe496a09-e4ec-4187-a856-7233935e5dae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:03:16.836162 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe496a09-e4ec-4187-a856-7233935e5dae service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:03:16.836447 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe496a09-e4ec-4187-a856-7233935e5dae service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:03:16.836729 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe496a09-e4ec-4187-a856-7233935e5dae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:03:16.846029 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe496a09-e4ec-4187-a856-7233935e5dae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:03:16.847490 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe496a09-e4ec-4187-a856-7233935e5dae service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:03:16.856676 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1439/2873] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 23:03:16 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 23:03:19.147712 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84949330-d6c7-451a-97e7-797ce47da559 service nova] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:03:19.149775 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-84949330-d6c7-451a-97e7-797ce47da559 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 23:03:19.167185 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84949330-d6c7-451a-97e7-797ce47da559 service nova] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:03:19.173286 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9398e263-fe23-4c55-86a5-7649cef1f2bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7829324db2574d38b18c749d0559177f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:03:19.174129 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9398e263-fe23-4c55-86a5-7649cef1f2bc service nova] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:03:19.174343 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9398e263-fe23-4c55-86a5-7649cef1f2bc service nova] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:03:19.174633 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9398e263-fe23-4c55-86a5-7649cef1f2bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:03:19.176422 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-9398e263-fe23-4c55-86a5-7649cef1f2bc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 23:03:19.194140 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9398e263-fe23-4c55-86a5-7649cef1f2bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:03:19.194140 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9398e263-fe23-4c55-86a5-7649cef1f2bc service nova] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:03:19.204268 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1435/2874] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 23:03:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 23:03:26.852618 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=6 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 23:03:29.198841 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=6 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 23:04:32.985849 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe496a09-e4ec-4187-a856-7233935e5dae service neutron] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:04:32.987236 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-fe496a09-e4ec-4187-a856-7233935e5dae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 23:04:33.073147 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe496a09-e4ec-4187-a856-7233935e5dae service neutron] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:04:33.078488 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f3a927-e544-4f37-954a-004a180c1208 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5500cebdd9d4f8e9aecb45d6c955878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:04:33.079208 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f3a927-e544-4f37-954a-004a180c1208 service placement] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:04:33.079501 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f3a927-e544-4f37-954a-004a180c1208 service placement] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:04:33.079784 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f3a927-e544-4f37-954a-004a180c1208 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:04:33.081545 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-26f3a927-e544-4f37-954a-004a180c1208 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 23:04:33.093323 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f3a927-e544-4f37-954a-004a180c1208 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:04:33.095028 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f3a927-e544-4f37-954a-004a180c1208 service placement] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:04:33.105254 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1440/2875] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 23:04:32 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 23:04:43.100899 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=37 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 23:05:27.519804 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9398e263-fe23-4c55-86a5-7649cef1f2bc service nova] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:05:27.559428 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9398e263-fe23-4c55-86a5-7649cef1f2bc service nova] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:05:27.563056 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffcc8a2-81c6-4dc5-93b3-bded75bb99fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:05:27.563700 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffcc8a2-81c6-4dc5-93b3-bded75bb99fb service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:05:27.563982 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffcc8a2-81c6-4dc5-93b3-bded75bb99fb service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:05:27.564259 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffcc8a2-81c6-4dc5-93b3-bded75bb99fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:05:27.566008 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-6ffcc8a2-81c6-4dc5-93b3-bded75bb99fb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 23:05:27.598443 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffcc8a2-81c6-4dc5-93b3-bded75bb99fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:05:27.599895 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffcc8a2-81c6-4dc5-93b3-bded75bb99fb service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:05:27.619550 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1436/2876] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 23:05:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 23:05:32.072689 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f3a927-e544-4f37-954a-004a180c1208 service placement] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:05:32.074147 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-26f3a927-e544-4f37-954a-004a180c1208 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 23:05:32.129864 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f3a927-e544-4f37-954a-004a180c1208 service placement] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:05:32.133496 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18786121-5488-4fd3-83aa-591afe0bb77c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecf4db289fba4e4a806198c62d31f7c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:05:32.134173 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18786121-5488-4fd3-83aa-591afe0bb77c service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:05:32.134462 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18786121-5488-4fd3-83aa-591afe0bb77c service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:05:32.134743 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18786121-5488-4fd3-83aa-591afe0bb77c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:05:32.136518 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-18786121-5488-4fd3-83aa-591afe0bb77c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 23:05:32.149716 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18786121-5488-4fd3-83aa-591afe0bb77c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:05:32.151165 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18786121-5488-4fd3-83aa-591afe0bb77c service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:05:32.162550 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1441/2877] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 23:05:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 23:05:37.616928 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=30 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 23:05:38.724569 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0580440e-006b-4350-8e05-5d054258f59e None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:05:38.725161 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0580440e-006b-4350-8e05-5d054258f59e None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:05:38.725161 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0580440e-006b-4350-8e05-5d054258f59e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:05:38.752925 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-0580440e-006b-4350-8e05-5d054258f59e None None] MFA Rules not processed for user `90915e1264324f16bc7bae7e46e0c97b`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 23:05:38.754375 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-0580440e-006b-4350-8e05-5d054258f59e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 23:05:38.761605 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1437/2878] 10.222.0.81 () {58 vars in 967 bytes} [Wed Jul 20 23:05:38 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 23:05:38.801656 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18786121-5488-4fd3-83aa-591afe0bb77c service cinder] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:05:38.812730 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18786121-5488-4fd3-83aa-591afe0bb77c service cinder] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:05:38.817719 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbdc5b9-4682-4f97-b100-688bef5115eb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '70e97436a08543a180225561ef8bc817', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:05:38.818302 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbdc5b9-4682-4f97-b100-688bef5115eb service swift] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:05:38.818585 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbdc5b9-4682-4f97-b100-688bef5115eb service swift] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:05:38.818866 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbdc5b9-4682-4f97-b100-688bef5115eb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:05:38.826977 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbdc5b9-4682-4f97-b100-688bef5115eb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:05:38.828570 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbdc5b9-4682-4f97-b100-688bef5115eb service swift] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:05:38.836454 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1442/2879] 10.222.0.81 () {60 vars in 1384 bytes} [Wed Jul 20 23:05:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 23:05:43.417461 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0580440e-006b-4350-8e05-5d054258f59e None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:05:43.430872 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0580440e-006b-4350-8e05-5d054258f59e None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:05:43.436177 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210cf538-a813-49de-a1bc-b9930ad9acdb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5500cebdd9d4f8e9aecb45d6c955878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:05:43.436835 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210cf538-a813-49de-a1bc-b9930ad9acdb service placement] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:05:43.437125 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210cf538-a813-49de-a1bc-b9930ad9acdb service placement] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:05:43.437422 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210cf538-a813-49de-a1bc-b9930ad9acdb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:05:43.439388 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-210cf538-a813-49de-a1bc-b9930ad9acdb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 23:05:43.451508 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210cf538-a813-49de-a1bc-b9930ad9acdb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:05:43.453116 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210cf538-a813-49de-a1bc-b9930ad9acdb service placement] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:05:43.463982 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1438/2880] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 23:05:43 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 23:05:47.005766 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbdc5b9-4682-4f97-b100-688bef5115eb service swift] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:05:47.084799 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbdc5b9-4682-4f97-b100-688bef5115eb service swift] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:05:47.089325 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e8407c-be0f-4215-8e2e-1c8f9482aa6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:05:47.090859 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e8407c-be0f-4215-8e2e-1c8f9482aa6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:05:47.090859 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e8407c-be0f-4215-8e2e-1c8f9482aa6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:05:47.090859 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e8407c-be0f-4215-8e2e-1c8f9482aa6d admin admin] PATH_INFO: `/v3/users/07feb13dc066470cae9d68d0d38d89cb` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:05:47.092478 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e8407c-be0f-4215-8e2e-1c8f9482aa6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=07feb13dc066470cae9d68d0d38d89cb)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:05:47.094381 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e8407c-be0f-4215-8e2e-1c8f9482aa6d admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:05:47.383628 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-97e8407c-be0f-4215-8e2e-1c8f9482aa6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07feb13dc066470cae9d68d0d38d89cb'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 23:05:47.384333 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-97e8407c-be0f-4215-8e2e-1c8f9482aa6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07feb13dc066470cae9d68d0d38d89cb'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 23:05:47.384942 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-97e8407c-be0f-4215-8e2e-1c8f9482aa6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07feb13dc066470cae9d68d0d38d89cb'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 23:05:47.392208 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-97e8407c-be0f-4215-8e2e-1c8f9482aa6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07feb13dc066470cae9d68d0d38d89cb'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 23:05:47.461465 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-97e8407c-be0f-4215-8e2e-1c8f9482aa6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07feb13dc066470cae9d68d0d38d89cb'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 23:05:47.469983 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 23:05:47.479763 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1443/2881] 10.222.0.81 () {58 vars in 1223 bytes} [Wed Jul 20 23:05:47 2022] DELETE /identity/v3/users/07feb13dc066470cae9d68d0d38d89cb => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 23:05:47.743558 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210cf538-a813-49de-a1bc-b9930ad9acdb service placement] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:05:47.757087 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210cf538-a813-49de-a1bc-b9930ad9acdb service placement] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:05:47.813834 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c07fd19-18e7-4cf7-8612-bc51fa1f3e33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:05:47.814715 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c07fd19-18e7-4cf7-8612-bc51fa1f3e33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:05:47.815029 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c07fd19-18e7-4cf7-8612-bc51fa1f3e33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:05:47.815315 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c07fd19-18e7-4cf7-8612-bc51fa1f3e33 admin admin] PATH_INFO: `/v3/projects/62f889bb65b048cb96e5548e4420df8f` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:05:47.816494 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c07fd19-18e7-4cf7-8612-bc51fa1f3e33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=62f889bb65b048cb96e5548e4420df8f)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:05:47.818432 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c07fd19-18e7-4cf7-8612-bc51fa1f3e33 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:05:47.907813 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-5c07fd19-18e7-4cf7-8612-bc51fa1f3e33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62f889bb65b048cb96e5548e4420df8f', 'request_id': 'req-5c07fd19-18e7-4cf7-8612-bc51fa1f3e33'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 23:05:48.021852 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-5c07fd19-18e7-4cf7-8612-bc51fa1f3e33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62f889bb65b048cb96e5548e4420df8f', 'request_id': 'req-5c07fd19-18e7-4cf7-8612-bc51fa1f3e33'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 23:05:48.028068 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-5c07fd19-18e7-4cf7-8612-bc51fa1f3e33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62f889bb65b048cb96e5548e4420df8f', 'request_id': 'req-5c07fd19-18e7-4cf7-8612-bc51fa1f3e33'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 23:05:48.051672 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-5c07fd19-18e7-4cf7-8612-bc51fa1f3e33 admin admin] The token cache is being invalidate because project 62f889bb65b048cb96e5548e4420df8f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 23:05:48.051956 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-5c07fd19-18e7-4cf7-8612-bc51fa1f3e33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 23:05:48.052579 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-5c07fd19-18e7-4cf7-8612-bc51fa1f3e33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 23:05:48.054140 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1439/2882] 10.222.0.81 () {58 vars in 1232 bytes} [Wed Jul 20 23:05:47 2022] DELETE /identity/v3/projects/62f889bb65b048cb96e5548e4420df8f => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 23:05:48.058178 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e8407c-be0f-4215-8e2e-1c8f9482aa6d admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:05:48.060182 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-97e8407c-be0f-4215-8e2e-1c8f9482aa6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 23:05:48.072935 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e8407c-be0f-4215-8e2e-1c8f9482aa6d admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:05:48.129068 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6d758a-7455-4460-bdff-570e841f1ed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:05:48.129958 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6d758a-7455-4460-bdff-570e841f1ed3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:05:48.130242 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6d758a-7455-4460-bdff-570e841f1ed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:05:48.130515 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6d758a-7455-4460-bdff-570e841f1ed3 admin admin] PATH_INFO: `/v3/users/6c4e16396f3f41dfacf0407220948e27` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:05:48.131874 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6d758a-7455-4460-bdff-570e841f1ed3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c4e16396f3f41dfacf0407220948e27)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:05:48.133839 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6d758a-7455-4460-bdff-570e841f1ed3 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:05:48.310418 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-7a6d758a-7455-4460-bdff-570e841f1ed3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c4e16396f3f41dfacf0407220948e27'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 23:05:48.311870 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-7a6d758a-7455-4460-bdff-570e841f1ed3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c4e16396f3f41dfacf0407220948e27'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 23:05:48.311870 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-7a6d758a-7455-4460-bdff-570e841f1ed3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c4e16396f3f41dfacf0407220948e27'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 23:05:48.318489 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-7a6d758a-7455-4460-bdff-570e841f1ed3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c4e16396f3f41dfacf0407220948e27'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 23:05:48.398896 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-7a6d758a-7455-4460-bdff-570e841f1ed3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c4e16396f3f41dfacf0407220948e27'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 23:05:48.417570 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1444/2883] 10.222.0.81 () {58 vars in 1223 bytes} [Wed Jul 20 23:05:48 2022] DELETE /identity/v3/users/6c4e16396f3f41dfacf0407220948e27 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 23:05:48.668713 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c07fd19-18e7-4cf7-8612-bc51fa1f3e33 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:05:48.670807 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-5c07fd19-18e7-4cf7-8612-bc51fa1f3e33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 23:05:48.684983 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c07fd19-18e7-4cf7-8612-bc51fa1f3e33 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:05:48.731642 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 23:05:48.741703 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974eb9d3-09f1-4aee-b7e3-bcdc92813725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:05:48.742579 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974eb9d3-09f1-4aee-b7e3-bcdc92813725 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:05:48.742860 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974eb9d3-09f1-4aee-b7e3-bcdc92813725 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:05:48.743243 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974eb9d3-09f1-4aee-b7e3-bcdc92813725 admin admin] PATH_INFO: `/v3/projects/59dfcc8c99e2485aab28879af32e2275` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:05:48.744409 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974eb9d3-09f1-4aee-b7e3-bcdc92813725 admin admin] RBAC: Authorizing `identity:delete_project(project_id=59dfcc8c99e2485aab28879af32e2275)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:05:48.746332 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974eb9d3-09f1-4aee-b7e3-bcdc92813725 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:05:48.849269 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-974eb9d3-09f1-4aee-b7e3-bcdc92813725 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59dfcc8c99e2485aab28879af32e2275', 'request_id': 'req-974eb9d3-09f1-4aee-b7e3-bcdc92813725'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 23:05:48.906829 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-974eb9d3-09f1-4aee-b7e3-bcdc92813725 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59dfcc8c99e2485aab28879af32e2275', 'request_id': 'req-974eb9d3-09f1-4aee-b7e3-bcdc92813725'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 23:05:48.912502 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-974eb9d3-09f1-4aee-b7e3-bcdc92813725 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59dfcc8c99e2485aab28879af32e2275', 'request_id': 'req-974eb9d3-09f1-4aee-b7e3-bcdc92813725'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 23:05:48.923911 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-974eb9d3-09f1-4aee-b7e3-bcdc92813725 admin admin] The token cache is being invalidate because project 59dfcc8c99e2485aab28879af32e2275 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80441) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 23:05:48.924198 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-974eb9d3-09f1-4aee-b7e3-bcdc92813725 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 23:05:48.924848 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-974eb9d3-09f1-4aee-b7e3-bcdc92813725 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 23:05:48.926377 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1440/2884] 10.222.0.81 () {58 vars in 1232 bytes} [Wed Jul 20 23:05:48 2022] DELETE /identity/v3/projects/59dfcc8c99e2485aab28879af32e2275 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 23:05:48.931499 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb97b69-8dda-4c9e-b20b-60755d2fb99f None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:05:48.931790 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb97b69-8dda-4c9e-b20b-60755d2fb99f None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:05:48.932067 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb97b69-8dda-4c9e-b20b-60755d2fb99f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:05:48.987842 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-ecb97b69-8dda-4c9e-b20b-60755d2fb99f None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 23:05:48.989056 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-ecb97b69-8dda-4c9e-b20b-60755d2fb99f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 23:05:49.020872 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 23:05:49.046366 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1445/2885] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 23:05:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 23:05:49.054487 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c52501-85ad-4ebe-a229-e04d744f4036 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:05:49.054777 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c52501-85ad-4ebe-a229-e04d744f4036 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:05:49.055088 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c52501-85ad-4ebe-a229-e04d744f4036 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:05:49.073165 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-54c52501-85ad-4ebe-a229-e04d744f4036 None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 23:05:49.074439 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-54c52501-85ad-4ebe-a229-e04d744f4036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 23:05:49.083082 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1441/2886] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 23:05:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 23:05:49.086871 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb97b69-8dda-4c9e-b20b-60755d2fb99f None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:05:49.097113 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb97b69-8dda-4c9e-b20b-60755d2fb99f None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:05:49.101868 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a65b37-f9b1-46b0-9916-02a1ab910a6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:05:49.102529 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a65b37-f9b1-46b0-9916-02a1ab910a6a admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:05:49.102817 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a65b37-f9b1-46b0-9916-02a1ab910a6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:05:49.103131 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a65b37-f9b1-46b0-9916-02a1ab910a6a admin admin] PATH_INFO: `/v3/domains` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:05:49.103856 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a65b37-f9b1-46b0-9916-02a1ab910a6a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80440) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 23:05:49.104165 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a65b37-f9b1-46b0-9916-02a1ab910a6a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:05:49.105749 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a65b37-f9b1-46b0-9916-02a1ab910a6a admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:05:49.117367 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1446/2887] 10.222.0.81 () {58 vars in 1152 bytes} [Wed Jul 20 23:05:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 23:05:49.121609 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c52501-85ad-4ebe-a229-e04d744f4036 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:05:49.131070 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c52501-85ad-4ebe-a229-e04d744f4036 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:05:49.135712 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8697ebe1-ec15-4545-b4ad-8921a39bdb5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:05:49.136435 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8697ebe1-ec15-4545-b4ad-8921a39bdb5c admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:05:49.136718 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8697ebe1-ec15-4545-b4ad-8921a39bdb5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:05:49.136995 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8697ebe1-ec15-4545-b4ad-8921a39bdb5c admin admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:05:49.137672 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8697ebe1-ec15-4545-b4ad-8921a39bdb5c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:05:49.140104 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8697ebe1-ec15-4545-b4ad-8921a39bdb5c admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:05:49.240547 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1442/2888] 10.222.0.81 () {60 vars in 1152 bytes} [Wed Jul 20 23:05:49 2022] POST /identity/v3/projects => generated 379 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 23:05:49.244773 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a65b37-f9b1-46b0-9916-02a1ab910a6a admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:05:49.255596 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a65b37-f9b1-46b0-9916-02a1ab910a6a admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:05:49.309354 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ea74c9-f398-4e58-bd8e-8361f06eb370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:05:49.310541 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ea74c9-f398-4e58-bd8e-8361f06eb370 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:05:49.310863 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ea74c9-f398-4e58-bd8e-8361f06eb370 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:05:49.311173 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ea74c9-f398-4e58-bd8e-8361f06eb370 admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:05:49.311869 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ea74c9-f398-4e58-bd8e-8361f06eb370 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:05:49.313761 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ea74c9-f398-4e58-bd8e-8361f06eb370 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:05:49.433432 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1447/2889] 10.222.0.81 () {60 vars in 1143 bytes} [Wed Jul 20 23:05:49 2022] POST /identity/v3/users => generated 290 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 23:05:49.438311 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8697ebe1-ec15-4545-b4ad-8921a39bdb5c admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:05:49.448720 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8697ebe1-ec15-4545-b4ad-8921a39bdb5c admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:05:49.453866 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f89610f-ad9a-461c-9170-a568c48a1958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:05:49.454524 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f89610f-ad9a-461c-9170-a568c48a1958 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:05:49.454823 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f89610f-ad9a-461c-9170-a568c48a1958 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:05:49.457002 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f89610f-ad9a-461c-9170-a568c48a1958 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:05:49.457002 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f89610f-ad9a-461c-9170-a568c48a1958 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:05:49.457510 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f89610f-ad9a-461c-9170-a568c48a1958 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:05:49.467720 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1443/2890] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 23:05:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 23:05:49.471501 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ea74c9-f398-4e58-bd8e-8361f06eb370 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:05:49.481336 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ea74c9-f398-4e58-bd8e-8361f06eb370 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:05:49.485924 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3a2dc5-a6ef-4365-92fb-e4991d8b9b75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:05:49.486585 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3a2dc5-a6ef-4365-92fb-e4991d8b9b75 admin admin] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:05:49.486872 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3a2dc5-a6ef-4365-92fb-e4991d8b9b75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:05:49.487192 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3a2dc5-a6ef-4365-92fb-e4991d8b9b75 admin admin] PATH_INFO: `/v3/roles` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:05:49.487915 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3a2dc5-a6ef-4365-92fb-e4991d8b9b75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:05:49.489484 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3a2dc5-a6ef-4365-92fb-e4991d8b9b75 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:05:49.500221 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1448/2891] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 23:05:49 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 23:05:49.504543 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f89610f-ad9a-461c-9170-a568c48a1958 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:05:49.514793 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f89610f-ad9a-461c-9170-a568c48a1958 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:05:49.519702 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9312ce04-00b7-4c05-99b1-0e265434f8a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:05:49.520594 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9312ce04-00b7-4c05-99b1-0e265434f8a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:05:49.520914 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9312ce04-00b7-4c05-99b1-0e265434f8a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:05:49.521197 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9312ce04-00b7-4c05-99b1-0e265434f8a2 admin admin] PATH_INFO: `/v3/projects/9044845bc9d34e1e8794a43dda688527/users/13390807045f4acdbbe4ea77af4dbb52/roles/559942c4de1744169f8b5e0404ecde6b` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:05:49.536254 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9312ce04-00b7-4c05-99b1-0e265434f8a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9044845bc9d34e1e8794a43dda688527, user_id=13390807045f4acdbbe4ea77af4dbb52, role_id=559942c4de1744169f8b5e0404ecde6b)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:05:49.543091 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9312ce04-00b7-4c05-99b1-0e265434f8a2 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:05:49.650222 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1444/2892] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 23:05:49 2022] PUT /identity/v3/projects/9044845bc9d34e1e8794a43dda688527/users/13390807045f4acdbbe4ea77af4dbb52/roles/559942c4de1744169f8b5e0404ecde6b => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 23:05:49.661531 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5c54ce-ae4e-481f-a231-3345c9efe76c None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:05:49.661831 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5c54ce-ae4e-481f-a231-3345c9efe76c None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:05:49.662119 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5c54ce-ae4e-481f-a231-3345c9efe76c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:05:49.681626 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-fd5c54ce-ae4e-481f-a231-3345c9efe76c None None] MFA Rules not processed for user `98f5d4c4be4845478aa0d31a1403d775`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 23:05:49.682837 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-fd5c54ce-ae4e-481f-a231-3345c9efe76c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 23:05:49.743078 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1449/2893] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 23:05:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 23:05:49.753264 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9312ce04-00b7-4c05-99b1-0e265434f8a2 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:05:49.755284 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-9312ce04-00b7-4c05-99b1-0e265434f8a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 23:05:49.824609 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9312ce04-00b7-4c05-99b1-0e265434f8a2 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:05:49.828233 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01c0aa8-1ec0-4f99-b91b-78ce5f425170 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:05:49.828876 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01c0aa8-1ec0-4f99-b91b-78ce5f425170 service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:05:49.829162 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01c0aa8-1ec0-4f99-b91b-78ce5f425170 service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:05:49.829446 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01c0aa8-1ec0-4f99-b91b-78ce5f425170 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:05:49.838439 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01c0aa8-1ec0-4f99-b91b-78ce5f425170 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:05:49.839896 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01c0aa8-1ec0-4f99-b91b-78ce5f425170 service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:05:49.844743 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 23:05:49.849424 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1445/2894] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 23:05:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 23:05:55.602715 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e803869d-4c4d-49e4-ae20-3a04eac89e12 None None] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:05:55.603252 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e803869d-4c4d-49e4-ae20-3a04eac89e12 None None] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:05:55.603356 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e803869d-4c4d-49e4-ae20-3a04eac89e12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:05:55.630099 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.auth.core [None req-e803869d-4c4d-49e4-ae20-3a04eac89e12 None None] MFA Rules not processed for user `13390807045f4acdbbe4ea77af4dbb52`. Rule list: `[]` (Enabled: `True`). {{(pid=80440) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 23:05:55.631380 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-e803869d-4c4d-49e4-ae20-3a04eac89e12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 23:05:55.664666 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 23:05:55.696204 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1450/2895] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 23:05:55 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 23:05:55.700744 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01c0aa8-1ec0-4f99-b91b-78ce5f425170 service neutron] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:05:55.710386 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01c0aa8-1ec0-4f99-b91b-78ce5f425170 service neutron] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:05:55.714939 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b3b8a1-1a2f-45a6-9009-c1d9ef7cf9ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:05:55.715729 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b3b8a1-1a2f-45a6-9009-c1d9ef7cf9ed admin admin] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:05:55.716023 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b3b8a1-1a2f-45a6-9009-c1d9ef7cf9ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:05:55.716311 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b3b8a1-1a2f-45a6-9009-c1d9ef7cf9ed admin admin] PATH_INFO: `/v3/projects` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:05:55.717006 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b3b8a1-1a2f-45a6-9009-c1d9ef7cf9ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:05:55.718933 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b3b8a1-1a2f-45a6-9009-c1d9ef7cf9ed admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:05:55.805595 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1446/2896] 10.222.0.81 () {60 vars in 1152 bytes} [Wed Jul 20 23:05:55 2022] POST /identity/v3/projects => generated 379 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 23:05:55.810387 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e803869d-4c4d-49e4-ae20-3a04eac89e12 None None] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:05:55.821615 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e803869d-4c4d-49e4-ae20-3a04eac89e12 None None] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:05:55.880260 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b3c16c-df81-4169-b924-0c8278701916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:05:55.880973 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b3c16c-df81-4169-b924-0c8278701916 admin admin] REQUEST_METHOD: `POST` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:05:55.881257 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b3c16c-df81-4169-b924-0c8278701916 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:05:55.881533 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b3c16c-df81-4169-b924-0c8278701916 admin admin] PATH_INFO: `/v3/users` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:05:55.882244 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b3c16c-df81-4169-b924-0c8278701916 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:05:55.884306 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b3c16c-df81-4169-b924-0c8278701916 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:05:55.983291 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1451/2897] 10.222.0.81 () {60 vars in 1143 bytes} [Wed Jul 20 23:05:55 2022] POST /identity/v3/users => generated 296 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 23:05:55.987248 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b3b8a1-1a2f-45a6-9009-c1d9ef7cf9ed admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:05:55.998067 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b3b8a1-1a2f-45a6-9009-c1d9ef7cf9ed admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:05:56.003331 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2b54a8-3cd9-4362-a0b3-d7b94ab8f961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:05:56.004330 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2b54a8-3cd9-4362-a0b3-d7b94ab8f961 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:05:56.004680 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2b54a8-3cd9-4362-a0b3-d7b94ab8f961 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:05:56.004964 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2b54a8-3cd9-4362-a0b3-d7b94ab8f961 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:05:56.005722 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2b54a8-3cd9-4362-a0b3-d7b94ab8f961 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:05:56.008384 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2b54a8-3cd9-4362-a0b3-d7b94ab8f961 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:05:56.018466 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1447/2898] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 23:05:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 23:05:56.022692 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b3c16c-df81-4169-b924-0c8278701916 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:05:56.033684 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b3c16c-df81-4169-b924-0c8278701916 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:05:56.038438 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3649461f-fff3-4464-9e8d-d4be40cc7d61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:05:56.039441 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3649461f-fff3-4464-9e8d-d4be40cc7d61 admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:05:56.039727 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3649461f-fff3-4464-9e8d-d4be40cc7d61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:05:56.040005 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3649461f-fff3-4464-9e8d-d4be40cc7d61 admin admin] PATH_INFO: `/v3/projects/8a5bc5c5e81144adb6bb5b630fd19b4f/users/65168bc3f48749d08967ca4e37b73d6d/roles/d89b6c92d35a401a8517f1d47f6be843` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:05:56.055221 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3649461f-fff3-4464-9e8d-d4be40cc7d61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a5bc5c5e81144adb6bb5b630fd19b4f, user_id=65168bc3f48749d08967ca4e37b73d6d, role_id=d89b6c92d35a401a8517f1d47f6be843)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:05:56.057921 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3649461f-fff3-4464-9e8d-d4be40cc7d61 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:05:56.171222 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1452/2899] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 23:05:56 2022] PUT /identity/v3/projects/8a5bc5c5e81144adb6bb5b630fd19b4f/users/65168bc3f48749d08967ca4e37b73d6d/roles/d89b6c92d35a401a8517f1d47f6be843 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 23:05:56.174577 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2b54a8-3cd9-4362-a0b3-d7b94ab8f961 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:05:56.184767 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2b54a8-3cd9-4362-a0b3-d7b94ab8f961 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:05:56.242727 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb0e3bb-da68-4cdd-9dbb-f56045863700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:05:56.243467 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb0e3bb-da68-4cdd-9dbb-f56045863700 admin admin] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:05:56.243761 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb0e3bb-da68-4cdd-9dbb-f56045863700 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:05:56.244047 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb0e3bb-da68-4cdd-9dbb-f56045863700 admin admin] PATH_INFO: `/v3/roles` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:05:56.244781 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb0e3bb-da68-4cdd-9dbb-f56045863700 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:05:56.246384 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb0e3bb-da68-4cdd-9dbb-f56045863700 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:05:56.257714 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1448/2900] 10.222.0.81 () {58 vars in 1121 bytes} [Wed Jul 20 23:05:56 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 23:05:56.262009 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3649461f-fff3-4464-9e8d-d4be40cc7d61 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:05:56.273223 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3649461f-fff3-4464-9e8d-d4be40cc7d61 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:05:56.278206 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36cff27-6def-40b0-88ac-25dd75e1110b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:05:56.279749 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36cff27-6def-40b0-88ac-25dd75e1110b admin admin] REQUEST_METHOD: `PUT` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:05:56.280226 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36cff27-6def-40b0-88ac-25dd75e1110b admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:05:56.280567 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36cff27-6def-40b0-88ac-25dd75e1110b admin admin] PATH_INFO: `/v3/projects/8a5bc5c5e81144adb6bb5b630fd19b4f/users/65168bc3f48749d08967ca4e37b73d6d/roles/d89b6c92d35a401a8517f1d47f6be843` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:05:56.282721 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36cff27-6def-40b0-88ac-25dd75e1110b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a5bc5c5e81144adb6bb5b630fd19b4f, user_id=65168bc3f48749d08967ca4e37b73d6d, role_id=d89b6c92d35a401a8517f1d47f6be843)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:05:56.285426 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36cff27-6def-40b0-88ac-25dd75e1110b admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:05:56.298214 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1453/2901] 10.222.0.81 () {60 vars in 1482 bytes} [Wed Jul 20 23:05:56 2022] PUT /identity/v3/projects/8a5bc5c5e81144adb6bb5b630fd19b4f/users/65168bc3f48749d08967ca4e37b73d6d/roles/d89b6c92d35a401a8517f1d47f6be843 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 23:06:01.987165 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6176379-e0ed-45ac-8755-0bc77f58ff54 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:06:01.987654 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6176379-e0ed-45ac-8755-0bc77f58ff54 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:06:01.987832 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6176379-e0ed-45ac-8755-0bc77f58ff54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:06:02.017797 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-b6176379-e0ed-45ac-8755-0bc77f58ff54 None None] MFA Rules not processed for user `65168bc3f48749d08967ca4e37b73d6d`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 23:06:02.019117 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-b6176379-e0ed-45ac-8755-0bc77f58ff54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 23:06:02.089757 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1449/2902] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 23:06:01 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 23:06:02.106069 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36cff27-6def-40b0-88ac-25dd75e1110b admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:06:02.108367 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-c36cff27-6def-40b0-88ac-25dd75e1110b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 23:06:02.187836 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36cff27-6def-40b0-88ac-25dd75e1110b admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:06:02.194719 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3047d5d8-eb3a-4f47-8f83-63c27cea025d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7829324db2574d38b18c749d0559177f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:06:02.195526 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3047d5d8-eb3a-4f47-8f83-63c27cea025d service nova] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:06:02.195816 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3047d5d8-eb3a-4f47-8f83-63c27cea025d service nova] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:06:02.196097 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3047d5d8-eb3a-4f47-8f83-63c27cea025d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:06:02.227040 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 23:06:02.258610 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3047d5d8-eb3a-4f47-8f83-63c27cea025d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:06:02.260337 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3047d5d8-eb3a-4f47-8f83-63c27cea025d service nova] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:06:02.269283 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1454/2903] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 23:06:02 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 23:06:12.086524 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=77,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 23:06:12.472773 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=16 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 23:06:27.052688 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a0fb49-f66a-49db-bb08-390b4578ca6f None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:06:27.053194 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a0fb49-f66a-49db-bb08-390b4578ca6f None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:06:27.053318 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a0fb49-f66a-49db-bb08-390b4578ca6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:06:27.070811 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-65a0fb49-f66a-49db-bb08-390b4578ca6f None None] MFA Rules not processed for user `13390807045f4acdbbe4ea77af4dbb52`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 23:06:27.072054 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-65a0fb49-f66a-49db-bb08-390b4578ca6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 23:06:27.079618 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1450/2904] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 23:06:27 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 23:06:27.394703 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3047d5d8-eb3a-4f47-8f83-63c27cea025d service nova] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:06:27.396167 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-3047d5d8-eb3a-4f47-8f83-63c27cea025d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 23:06:27.450876 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3047d5d8-eb3a-4f47-8f83-63c27cea025d service nova] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:06:27.455499 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bf98b0-a57a-4a5a-ab77-c0ada8d8047c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:06:27.456080 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bf98b0-a57a-4a5a-ab77-c0ada8d8047c service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:06:27.456365 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bf98b0-a57a-4a5a-ab77-c0ada8d8047c service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:06:27.456638 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bf98b0-a57a-4a5a-ab77-c0ada8d8047c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:06:27.458359 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-82bf98b0-a57a-4a5a-ab77-c0ada8d8047c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 23:06:27.467791 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bf98b0-a57a-4a5a-ab77-c0ada8d8047c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:06:27.469188 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bf98b0-a57a-4a5a-ab77-c0ada8d8047c service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:06:27.476980 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1455/2905] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 23:06:27 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 23:06:37.065414 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=3 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 23:06:37.473596 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=26 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 23:06:43.794294 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a0fb49-f66a-49db-bb08-390b4578ca6f None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:06:43.795852 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-65a0fb49-f66a-49db-bb08-390b4578ca6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 23:06:43.812050 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a0fb49-f66a-49db-bb08-390b4578ca6f None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:06:43.817991 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514dcd22-db58-4334-b675-0e403f9f2bca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7829324db2574d38b18c749d0559177f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:06:43.818638 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514dcd22-db58-4334-b675-0e403f9f2bca service nova] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:06:43.818927 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514dcd22-db58-4334-b675-0e403f9f2bca service nova] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:06:43.819244 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514dcd22-db58-4334-b675-0e403f9f2bca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:06:43.821039 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-514dcd22-db58-4334-b675-0e403f9f2bca service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 23:06:43.833705 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514dcd22-db58-4334-b675-0e403f9f2bca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:06:43.835317 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514dcd22-db58-4334-b675-0e403f9f2bca service nova] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:06:43.846293 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1451/2906] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 23:06:43 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 23:06:47.178166 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bf98b0-a57a-4a5a-ab77-c0ada8d8047c service neutron] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:06:47.186886 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bf98b0-a57a-4a5a-ab77-c0ada8d8047c service neutron] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:06:47.190637 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c014a2-fe4f-40f5-b962-6a9b252bcdf2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:06:47.191314 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c014a2-fe4f-40f5-b962-6a9b252bcdf2 service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:06:47.191596 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c014a2-fe4f-40f5-b962-6a9b252bcdf2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:06:47.191874 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c014a2-fe4f-40f5-b962-6a9b252bcdf2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:06:47.200912 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c014a2-fe4f-40f5-b962-6a9b252bcdf2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:06:47.202324 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c014a2-fe4f-40f5-b962-6a9b252bcdf2 service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:06:47.212126 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1456/2907] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 23:06:47 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 23:06:53.841371 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=6 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 23:06:57.207981 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=6 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 23:07:44.667831 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514dcd22-db58-4334-b675-0e403f9f2bca service nova] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:07:44.677089 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514dcd22-db58-4334-b675-0e403f9f2bca service nova] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:07:44.682444 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50203503-9aa4-4337-b86e-0a31b2b25bd0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:07:44.683619 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50203503-9aa4-4337-b86e-0a31b2b25bd0 service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:07:44.683899 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50203503-9aa4-4337-b86e-0a31b2b25bd0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:07:44.684183 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50203503-9aa4-4337-b86e-0a31b2b25bd0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:07:44.685952 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-50203503-9aa4-4337-b86e-0a31b2b25bd0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 23:07:44.695509 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50203503-9aa4-4337-b86e-0a31b2b25bd0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:07:44.698000 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50203503-9aa4-4337-b86e-0a31b2b25bd0 service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:07:44.705762 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1452/2908] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 23:07:44 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 23:07:54.702118 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=6 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 23:09:06.229351 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c014a2-fe4f-40f5-b962-6a9b252bcdf2 service neutron] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:09:06.242223 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c014a2-fe4f-40f5-b962-6a9b252bcdf2 service neutron] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:09:06.248137 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce39eb3a-ceed-42b0-99f5-fd5fe20e7473 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7829324db2574d38b18c749d0559177f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:09:06.249391 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce39eb3a-ceed-42b0-99f5-fd5fe20e7473 service nova] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:09:06.249685 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce39eb3a-ceed-42b0-99f5-fd5fe20e7473 service nova] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:09:06.249966 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce39eb3a-ceed-42b0-99f5-fd5fe20e7473 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:09:06.251751 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-ce39eb3a-ceed-42b0-99f5-fd5fe20e7473 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 23:09:06.266047 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce39eb3a-ceed-42b0-99f5-fd5fe20e7473 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:09:06.267666 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce39eb3a-ceed-42b0-99f5-fd5fe20e7473 service nova] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:09:06.279869 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1457/2909] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 23:09:06 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 23:09:08.116700 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881cf8a7-dd1c-49fd-8de7-dd5943bafe8e None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:09:08.117177 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881cf8a7-dd1c-49fd-8de7-dd5943bafe8e None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:09:08.117288 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881cf8a7-dd1c-49fd-8de7-dd5943bafe8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:09:08.135562 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-881cf8a7-dd1c-49fd-8de7-dd5943bafe8e None None] MFA Rules not processed for user `13390807045f4acdbbe4ea77af4dbb52`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 23:09:08.137567 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-881cf8a7-dd1c-49fd-8de7-dd5943bafe8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 23:09:08.144425 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1453/2910] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 23:09:08 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 23:09:08.455752 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce39eb3a-ceed-42b0-99f5-fd5fe20e7473 service nova] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:09:08.465092 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce39eb3a-ceed-42b0-99f5-fd5fe20e7473 service nova] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:09:08.468676 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c2f868-01f1-4a36-b038-8d85fb7861d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:09:08.469342 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c2f868-01f1-4a36-b038-8d85fb7861d9 service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:09:08.469650 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c2f868-01f1-4a36-b038-8d85fb7861d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:09:08.469926 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c2f868-01f1-4a36-b038-8d85fb7861d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:09:08.471669 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-64c2f868-01f1-4a36-b038-8d85fb7861d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 23:09:08.481598 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c2f868-01f1-4a36-b038-8d85fb7861d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:09:08.483033 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c2f868-01f1-4a36-b038-8d85fb7861d9 service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:09:08.490849 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1458/2911] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 23:09:08 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 23:09:09.280912 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881cf8a7-dd1c-49fd-8de7-dd5943bafe8e None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:09:09.282382 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-881cf8a7-dd1c-49fd-8de7-dd5943bafe8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 23:09:09.355917 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881cf8a7-dd1c-49fd-8de7-dd5943bafe8e None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:09:09.360954 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9eb92a5-ce8c-4613-a035-1c36bfcbcd4e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5500cebdd9d4f8e9aecb45d6c955878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:09:09.361645 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eb92a5-ce8c-4613-a035-1c36bfcbcd4e service placement] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:09:09.361932 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eb92a5-ce8c-4613-a035-1c36bfcbcd4e service placement] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:09:09.362218 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eb92a5-ce8c-4613-a035-1c36bfcbcd4e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:09:09.364090 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-a9eb92a5-ce8c-4613-a035-1c36bfcbcd4e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 23:09:09.376928 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9eb92a5-ce8c-4613-a035-1c36bfcbcd4e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:09:09.378425 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9eb92a5-ce8c-4613-a035-1c36bfcbcd4e service placement] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:09:09.396988 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1454/2912] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 23:09:09 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 23:09:18.487478 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=12 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 23:09:19.384442 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=35 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 23:09:30.499928 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c2f868-01f1-4a36-b038-8d85fb7861d9 service neutron] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:09:30.513938 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c2f868-01f1-4a36-b038-8d85fb7861d9 service neutron] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:09:30.519643 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202a62e0-5bbb-4804-a4c6-40b7e7f2fed3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7829324db2574d38b18c749d0559177f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:09:30.520292 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202a62e0-5bbb-4804-a4c6-40b7e7f2fed3 service nova] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:09:30.520585 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202a62e0-5bbb-4804-a4c6-40b7e7f2fed3 service nova] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:09:30.520911 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202a62e0-5bbb-4804-a4c6-40b7e7f2fed3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:09:30.522714 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-202a62e0-5bbb-4804-a4c6-40b7e7f2fed3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 23:09:30.535337 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202a62e0-5bbb-4804-a4c6-40b7e7f2fed3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:09:30.536914 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202a62e0-5bbb-4804-a4c6-40b7e7f2fed3 service nova] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:09:30.547629 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1459/2913] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 23:09:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 23:09:37.951936 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9eb92a5-ce8c-4613-a035-1c36bfcbcd4e service placement] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:09:37.963306 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9eb92a5-ce8c-4613-a035-1c36bfcbcd4e service placement] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:09:37.968478 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f7e8f7-4b5a-467f-9b11-562dc78cba99 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5500cebdd9d4f8e9aecb45d6c955878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:09:37.969119 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f7e8f7-4b5a-467f-9b11-562dc78cba99 service placement] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:09:37.969408 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f7e8f7-4b5a-467f-9b11-562dc78cba99 service placement] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:09:37.969692 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f7e8f7-4b5a-467f-9b11-562dc78cba99 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:09:37.971523 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-55f7e8f7-4b5a-467f-9b11-562dc78cba99 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 23:09:37.983700 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f7e8f7-4b5a-467f-9b11-562dc78cba99 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:09:37.985272 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f7e8f7-4b5a-467f-9b11-562dc78cba99 service placement] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:09:37.995757 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1455/2914] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 23:09:37 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 23:09:40.543186 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=6 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 23:09:47.991485 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=6 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 23:11:16.921043 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202a62e0-5bbb-4804-a4c6-40b7e7f2fed3 service nova] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:11:16.922441 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-202a62e0-5bbb-4804-a4c6-40b7e7f2fed3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 23:11:17.003084 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202a62e0-5bbb-4804-a4c6-40b7e7f2fed3 service nova] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:11:17.006686 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3520014f-0f06-4e89-b036-b73afbd6c638 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecf4db289fba4e4a806198c62d31f7c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:11:17.007366 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3520014f-0f06-4e89-b036-b73afbd6c638 service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:11:17.007655 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3520014f-0f06-4e89-b036-b73afbd6c638 service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:11:17.007931 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3520014f-0f06-4e89-b036-b73afbd6c638 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:11:17.016165 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3520014f-0f06-4e89-b036-b73afbd6c638 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:11:17.018169 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3520014f-0f06-4e89-b036-b73afbd6c638 service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:11:17.026757 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1460/2915] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 23:11:16 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 23:11:27.022283 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=35 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 23:11:28.491459 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c93692-2b0e-4214-a9f6-f1f9e501f884 None None] REQUEST_METHOD: `POST` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:11:28.498088 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c93692-2b0e-4214-a9f6-f1f9e501f884 None None] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:11:28.498088 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c93692-2b0e-4214-a9f6-f1f9e501f884 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:11:28.511540 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.auth.core [None req-13c93692-2b0e-4214-a9f6-f1f9e501f884 None None] MFA Rules not processed for user `13390807045f4acdbbe4ea77af4dbb52`. Rule list: `[]` (Enabled: `True`). {{(pid=80441) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 23:11:28.512752 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-13c93692-2b0e-4214-a9f6-f1f9e501f884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 23:11:28.519887 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1456/2916] 10.222.0.81 () {58 vars in 957 bytes} [Wed Jul 20 23:11:28 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 23:11:28.556593 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3520014f-0f06-4e89-b036-b73afbd6c638 service cinder] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:11:28.565205 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3520014f-0f06-4e89-b036-b73afbd6c638 service cinder] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:11:28.568937 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77455a5-4fbd-4512-bd4a-9cc77aae8033 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ecf4db289fba4e4a806198c62d31f7c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:11:28.569578 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77455a5-4fbd-4512-bd4a-9cc77aae8033 service cinder] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:11:28.569864 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77455a5-4fbd-4512-bd4a-9cc77aae8033 service cinder] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:11:28.570146 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77455a5-4fbd-4512-bd4a-9cc77aae8033 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:11:28.572505 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-b77455a5-4fbd-4512-bd4a-9cc77aae8033 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 23:11:28.585875 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77455a5-4fbd-4512-bd4a-9cc77aae8033 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:11:28.587311 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77455a5-4fbd-4512-bd4a-9cc77aae8033 service cinder] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:11:28.598301 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1461/2917] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 23:11:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 23:11:29.606016 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c93692-2b0e-4214-a9f6-f1f9e501f884 None None] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:11:29.616464 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c93692-2b0e-4214-a9f6-f1f9e501f884 None None] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:11:29.620511 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124f4d36-5d15-4df0-8550-f6e861cea1ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:11:29.621157 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124f4d36-5d15-4df0-8550-f6e861cea1ec service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:11:29.621440 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124f4d36-5d15-4df0-8550-f6e861cea1ec service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:11:29.621967 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124f4d36-5d15-4df0-8550-f6e861cea1ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:11:29.623808 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-124f4d36-5d15-4df0-8550-f6e861cea1ec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 23:11:29.638316 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124f4d36-5d15-4df0-8550-f6e861cea1ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:11:29.639784 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124f4d36-5d15-4df0-8550-f6e861cea1ec service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:11:29.652425 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1457/2918] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 23:11:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 23:11:38.593272 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=6 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 23:11:39.646308 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=9 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 23:11:45.311387 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77455a5-4fbd-4512-bd4a-9cc77aae8033 service cinder] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:11:45.324086 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77455a5-4fbd-4512-bd4a-9cc77aae8033 service cinder] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:11:45.328178 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5015fd6e-b7b8-49cf-b054-fe5f70836432 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7829324db2574d38b18c749d0559177f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:11:45.329165 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5015fd6e-b7b8-49cf-b054-fe5f70836432 service nova] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:11:45.329803 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5015fd6e-b7b8-49cf-b054-fe5f70836432 service nova] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:11:45.330258 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5015fd6e-b7b8-49cf-b054-fe5f70836432 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:11:45.339882 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5015fd6e-b7b8-49cf-b054-fe5f70836432 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:11:45.341449 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5015fd6e-b7b8-49cf-b054-fe5f70836432 service nova] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:11:45.352422 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1462/2919] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 23:11:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 23:11:48.576293 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124f4d36-5d15-4df0-8550-f6e861cea1ec service neutron] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:11:48.585004 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124f4d36-5d15-4df0-8550-f6e861cea1ec service neutron] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:11:48.588655 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350b1ac0-82bb-4b16-8e22-1ced13dd67e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:11:48.589325 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350b1ac0-82bb-4b16-8e22-1ced13dd67e1 service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:11:48.589610 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350b1ac0-82bb-4b16-8e22-1ced13dd67e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:11:48.589920 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350b1ac0-82bb-4b16-8e22-1ced13dd67e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:11:48.599585 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350b1ac0-82bb-4b16-8e22-1ced13dd67e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:11:48.600945 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350b1ac0-82bb-4b16-8e22-1ced13dd67e1 service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:11:48.611077 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1458/2920] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 23:11:48 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 23:11:55.347383 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=6 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 23:11:58.606196 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=6 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 23:12:48.668177 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5015fd6e-b7b8-49cf-b054-fe5f70836432 service nova] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:12:48.677443 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5015fd6e-b7b8-49cf-b054-fe5f70836432 service nova] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:12:48.680983 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3211b7-8b82-4286-abcb-a09d9c7da309 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:12:48.681625 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3211b7-8b82-4286-abcb-a09d9c7da309 service neutron] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:12:48.681910 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3211b7-8b82-4286-abcb-a09d9c7da309 service neutron] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:12:48.682185 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3211b7-8b82-4286-abcb-a09d9c7da309 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:12:48.689467 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3211b7-8b82-4286-abcb-a09d9c7da309 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:12:48.690881 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3211b7-8b82-4286-abcb-a09d9c7da309 service neutron] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:12:48.699477 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1463/2921] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 23:12:48 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 32 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 23:12:58.695769 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=6 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 23:13:56.884545 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350b1ac0-82bb-4b16-8e22-1ced13dd67e1 service neutron] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:13:56.896209 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350b1ac0-82bb-4b16-8e22-1ced13dd67e1 service neutron] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:13:56.899958 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97deb6c1-c03a-4df9-8624-7c37b1360f7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '333c2b1b11cc4c478abfd965a72e9d66', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:13:56.902171 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97deb6c1-c03a-4df9-8624-7c37b1360f7f service neutron] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:13:56.902171 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97deb6c1-c03a-4df9-8624-7c37b1360f7f service neutron] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:13:56.902171 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97deb6c1-c03a-4df9-8624-7c37b1360f7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:13:56.961880 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97deb6c1-c03a-4df9-8624-7c37b1360f7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:13:56.963361 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97deb6c1-c03a-4df9-8624-7c37b1360f7f service neutron] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:13:56.972852 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1459/2922] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 23:13:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 23:14:00.475605 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3211b7-8b82-4286-abcb-a09d9c7da309 service neutron] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:14:00.477542 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-7f3211b7-8b82-4286-abcb-a09d9c7da309 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 23:14:00.493297 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3211b7-8b82-4286-abcb-a09d9c7da309 service neutron] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:14:00.498284 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd853f2-785e-419e-80cd-4224e2bee1f6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5500cebdd9d4f8e9aecb45d6c955878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:14:00.498924 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd853f2-785e-419e-80cd-4224e2bee1f6 service placement] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:14:00.502478 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd853f2-785e-419e-80cd-4224e2bee1f6 service placement] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:14:00.502478 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd853f2-785e-419e-80cd-4224e2bee1f6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:14:00.502478 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-abd853f2-785e-419e-80cd-4224e2bee1f6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 23:14:00.513290 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd853f2-785e-419e-80cd-4224e2bee1f6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:14:00.514889 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd853f2-785e-419e-80cd-4224e2bee1f6 service placement] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:14:00.525170 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1464/2923] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 23:14:00 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 23:14:03.946063 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97deb6c1-c03a-4df9-8624-7c37b1360f7f service neutron] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:14:03.957018 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97deb6c1-c03a-4df9-8624-7c37b1360f7f service neutron] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:14:03.961773 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9263aafd-1823-4105-ad84-dacf134af282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:14:03.962689 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9263aafd-1823-4105-ad84-dacf134af282 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:14:03.962990 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9263aafd-1823-4105-ad84-dacf134af282 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:14:03.963307 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9263aafd-1823-4105-ad84-dacf134af282 admin admin] PATH_INFO: `/v3/users/13390807045f4acdbbe4ea77af4dbb52` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:14:03.964669 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9263aafd-1823-4105-ad84-dacf134af282 admin admin] RBAC: Authorizing `identity:delete_user(user_id=13390807045f4acdbbe4ea77af4dbb52)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:14:03.968130 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9263aafd-1823-4105-ad84-dacf134af282 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:14:04.175781 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-9263aafd-1823-4105-ad84-dacf134af282 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13390807045f4acdbbe4ea77af4dbb52'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 23:14:04.176535 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-9263aafd-1823-4105-ad84-dacf134af282 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13390807045f4acdbbe4ea77af4dbb52'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 23:14:04.177138 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-9263aafd-1823-4105-ad84-dacf134af282 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13390807045f4acdbbe4ea77af4dbb52'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 23:14:04.184844 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-9263aafd-1823-4105-ad84-dacf134af282 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13390807045f4acdbbe4ea77af4dbb52'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 23:14:04.253387 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-9263aafd-1823-4105-ad84-dacf134af282 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13390807045f4acdbbe4ea77af4dbb52'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 23:14:04.272643 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1460/2924] 10.222.0.81 () {58 vars in 1223 bytes} [Wed Jul 20 23:14:03 2022] DELETE /identity/v3/users/13390807045f4acdbbe4ea77af4dbb52 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 23:14:04.553061 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd853f2-785e-419e-80cd-4224e2bee1f6 service placement] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:14:04.563670 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd853f2-785e-419e-80cd-4224e2bee1f6 service placement] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:14:04.631102 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424c35eb-2ae8-4e50-b15f-b96961d1e931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:14:04.632170 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424c35eb-2ae8-4e50-b15f-b96961d1e931 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:14:04.632457 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424c35eb-2ae8-4e50-b15f-b96961d1e931 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:14:04.632733 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424c35eb-2ae8-4e50-b15f-b96961d1e931 admin admin] PATH_INFO: `/v3/projects/9044845bc9d34e1e8794a43dda688527` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:14:04.634122 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424c35eb-2ae8-4e50-b15f-b96961d1e931 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9044845bc9d34e1e8794a43dda688527)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:14:04.636429 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424c35eb-2ae8-4e50-b15f-b96961d1e931 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:14:04.762786 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-424c35eb-2ae8-4e50-b15f-b96961d1e931 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9044845bc9d34e1e8794a43dda688527', 'request_id': 'req-424c35eb-2ae8-4e50-b15f-b96961d1e931'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 23:14:04.769534 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-424c35eb-2ae8-4e50-b15f-b96961d1e931 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9044845bc9d34e1e8794a43dda688527', 'request_id': 'req-424c35eb-2ae8-4e50-b15f-b96961d1e931'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 23:14:04.863413 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-424c35eb-2ae8-4e50-b15f-b96961d1e931 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9044845bc9d34e1e8794a43dda688527', 'request_id': 'req-424c35eb-2ae8-4e50-b15f-b96961d1e931'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 23:14:04.874115 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-424c35eb-2ae8-4e50-b15f-b96961d1e931 admin admin] The token cache is being invalidate because project 9044845bc9d34e1e8794a43dda688527 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 23:14:04.874407 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-424c35eb-2ae8-4e50-b15f-b96961d1e931 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 23:14:04.875159 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-424c35eb-2ae8-4e50-b15f-b96961d1e931 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 23:14:04.876606 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1465/2925] 10.222.0.81 () {58 vars in 1232 bytes} [Wed Jul 20 23:14:04 2022] DELETE /identity/v3/projects/9044845bc9d34e1e8794a43dda688527 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 23:14:04.880632 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9263aafd-1823-4105-ad84-dacf134af282 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:14:04.882060 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-9263aafd-1823-4105-ad84-dacf134af282 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 23:14:04.895608 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9263aafd-1823-4105-ad84-dacf134af282 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:14:04.952547 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0d5c9b-b834-4318-8dec-82e3cb866cd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:14:04.953430 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0d5c9b-b834-4318-8dec-82e3cb866cd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:14:04.953717 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0d5c9b-b834-4318-8dec-82e3cb866cd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:14:04.953997 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0d5c9b-b834-4318-8dec-82e3cb866cd2 admin admin] PATH_INFO: `/v3/users/65168bc3f48749d08967ca4e37b73d6d` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:14:04.955233 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0d5c9b-b834-4318-8dec-82e3cb866cd2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=65168bc3f48749d08967ca4e37b73d6d)` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:14:04.957793 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0d5c9b-b834-4318-8dec-82e3cb866cd2 admin admin] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:14:04.963214 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 23:14:05.147065 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-1d0d5c9b-b834-4318-8dec-82e3cb866cd2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65168bc3f48749d08967ca4e37b73d6d'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 23:14:05.147648 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-1d0d5c9b-b834-4318-8dec-82e3cb866cd2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65168bc3f48749d08967ca4e37b73d6d'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 23:14:05.148254 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-1d0d5c9b-b834-4318-8dec-82e3cb866cd2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65168bc3f48749d08967ca4e37b73d6d'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 23:14:05.154928 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-1d0d5c9b-b834-4318-8dec-82e3cb866cd2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65168bc3f48749d08967ca4e37b73d6d'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 23:14:05.230368 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.notifications [None req-1d0d5c9b-b834-4318-8dec-82e3cb866cd2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65168bc3f48749d08967ca4e37b73d6d'} {{(pid=80441) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 23:14:05.249430 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1461/2926] 10.222.0.81 () {58 vars in 1223 bytes} [Wed Jul 20 23:14:04 2022] DELETE /identity/v3/users/65168bc3f48749d08967ca4e37b73d6d => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 23:14:05.494524 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424c35eb-2ae8-4e50-b15f-b96961d1e931 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:14:05.496063 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-424c35eb-2ae8-4e50-b15f-b96961d1e931 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 23:14:05.511103 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424c35eb-2ae8-4e50-b15f-b96961d1e931 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:14:05.568592 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69317fa4-659d-4495-9b3e-f94e28280201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98f5d4c4be4845478aa0d31a1403d775', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '015b89c7582349b2a89f8bbaea747017', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:14:05.569752 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69317fa4-659d-4495-9b3e-f94e28280201 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:14:05.569900 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69317fa4-659d-4495-9b3e-f94e28280201 admin admin] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:14:05.570100 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69317fa4-659d-4495-9b3e-f94e28280201 admin admin] PATH_INFO: `/v3/projects/8a5bc5c5e81144adb6bb5b630fd19b4f` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:14:05.571411 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69317fa4-659d-4495-9b3e-f94e28280201 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a5bc5c5e81144adb6bb5b630fd19b4f)` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:14:05.573331 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69317fa4-659d-4495-9b3e-f94e28280201 admin admin] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:14:05.594777 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 23:14:05.652383 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-69317fa4-659d-4495-9b3e-f94e28280201 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a5bc5c5e81144adb6bb5b630fd19b4f', 'request_id': 'req-69317fa4-659d-4495-9b3e-f94e28280201'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 23:14:05.658962 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-69317fa4-659d-4495-9b3e-f94e28280201 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a5bc5c5e81144adb6bb5b630fd19b4f', 'request_id': 'req-69317fa4-659d-4495-9b3e-f94e28280201'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 23:14:05.699070 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-69317fa4-659d-4495-9b3e-f94e28280201 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a5bc5c5e81144adb6bb5b630fd19b4f', 'request_id': 'req-69317fa4-659d-4495-9b3e-f94e28280201'} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 23:14:05.713450 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-69317fa4-659d-4495-9b3e-f94e28280201 admin admin] The token cache is being invalidate because project 8a5bc5c5e81144adb6bb5b630fd19b4f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80440) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 23:14:05.713740 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-69317fa4-659d-4495-9b3e-f94e28280201 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 23:14:05.714396 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.notifications [None req-69317fa4-659d-4495-9b3e-f94e28280201 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80440) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 23:14:05.716019 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1466/2927] 10.222.0.81 () {58 vars in 1232 bytes} [Wed Jul 20 23:14:05 2022] DELETE /identity/v3/projects/8a5bc5c5e81144adb6bb5b630fd19b4f => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 23:14:15.245822 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 23:14:15.710537 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG dbcounter [-] [80440] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80440) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 23:14:41.924534 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0d5c9b-b834-4318-8dec-82e3cb866cd2 admin admin] Authenticating user token {{(pid=80441) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:14:41.926021 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-1d0d5c9b-b834-4318-8dec-82e3cb866cd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 23:14:42.045109 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0d5c9b-b834-4318-8dec-82e3cb866cd2 admin admin] Validating token access rules against request {{(pid=80441) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:14:42.103363 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036db370-9852-4b14-b0a8-ede48f2a7b7f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5500cebdd9d4f8e9aecb45d6c955878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80441) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:14:42.104020 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036db370-9852-4b14-b0a8-ede48f2a7b7f service placement] REQUEST_METHOD: `GET` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:14:42.104308 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036db370-9852-4b14-b0a8-ede48f2a7b7f service placement] SCRIPT_NAME: `/identity` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:14:42.104591 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036db370-9852-4b14-b0a8-ede48f2a7b7f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80441) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:14:42.106378 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.fernet_utils [None req-036db370-9852-4b14-b0a8-ede48f2a7b7f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80441) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 23:14:42.119045 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036db370-9852-4b14-b0a8-ede48f2a7b7f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:14:42.120645 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036db370-9852-4b14-b0a8-ede48f2a7b7f service placement] RBAC: Authorization granted {{(pid=80441) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:14:42.131254 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: [pid: 80441|app: 0|req: 1462/2928] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 23:14:41 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 23:14:52.126299 dv-cinder-799871-38-iscsi devstack@keystone.service[80441]: DEBUG dbcounter [-] [80441] Writing DB stats keystone:SELECT=80 {{(pid=80441) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 23:19:44.878837 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69317fa4-659d-4495-9b3e-f94e28280201 admin admin] Authenticating user token {{(pid=80440) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 23:19:44.880295 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.fernet_utils [None req-69317fa4-659d-4495-9b3e-f94e28280201 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80440) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 23:19:44.936417 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69317fa4-659d-4495-9b3e-f94e28280201 admin admin] Validating token access rules against request {{(pid=80440) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 23:19:44.941592 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d3c523-0c51-4e73-90d2-e52d83646dd7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5500cebdd9d4f8e9aecb45d6c955878', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '385edf2e59e546d090ace36600e29032', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80440) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 23:19:44.942251 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d3c523-0c51-4e73-90d2-e52d83646dd7 service placement] REQUEST_METHOD: `GET` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 23:19:44.942539 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d3c523-0c51-4e73-90d2-e52d83646dd7 service placement] SCRIPT_NAME: `/identity` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 23:19:44.942823 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d3c523-0c51-4e73-90d2-e52d83646dd7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80440) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 23:19:44.952229 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d3c523-0c51-4e73-90d2-e52d83646dd7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 23:19:44.953804 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d3c523-0c51-4e73-90d2-e52d83646dd7 service placement] RBAC: Authorization granted {{(pid=80440) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 23:19:44.963829 dv-cinder-799871-38-iscsi devstack@keystone.service[80440]: [pid: 80440|app: 0|req: 1467/2929] 10.222.0.81 () {60 vars in 1365 bytes} [Wed Jul 20 23:19:44 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0)